Leveraging process mining to drive Power Platform performance to do more is the topic of this episode with Host Joel Lindstrom and low code expert Amarnath Vootla.
Process Mining analyzes system log files to discover bottlenecks, variants, and optimization opportunities in business processes. It's an AI powered X-ray into your workflows and efficiency rating. This can be refreshed on an ongoing basis to provide near-real-time monitoring of processes to identify inefficiencies and failure points.
The result? Better insight and prioritized decision-making around automation and optimization with Microsoft Power Platform
To learn more, check out our September edition of Power Platform INSIGHTS !
global.hitachi-solutions.com
Leveraging process mining to drive Power Platform performance to do more is the topic of this episode with Host Joel Lindstrom and low code expert Amarnath Vootla.
Process Mining analyzes system log files to discover bottlenecks, variants, and optimization opportunities in business processes. It's an AI powered X-ray into your workflows and efficiency rating. This can be refreshed on an ongoing basis to provide near-real-time monitoring of processes to identify inefficiencies and failure points.
The result? Better insight and prioritized decision-making around automation and optimization with Microsoft Power Platform
To learn more, check out our September edition of Power Platform INSIGHTS !
global.hitachi-solutions.com
Mon, Aug 28, 2023 3:41PM • 33:16
EPISODE TITLE:
SUMMARY KEYWORDS
process, data, mining, team, work, automate, advisor, power, logs, flow, system, hitachi solutions, Microsoft, Power Platform, view, rework, automation, insights, notifications, analyze, task, business advisory services
SPEAKERS
Amarnath Vootla, Joel Lindstrom from Hitachi Solutions
JL: Welcome to the Exchanges podcast from Hitachi Solutions. I'm Joel Lindstrom. And joining me today is Mr. Amarnath Vootla, who's an architect for Automation here on the Rapid Solution Development team. Today, we're going to be talking about some exciting new things that are available as part of as part of power automate, how to leverage process mining to drive your process excellence in digital transformation and automation.
And so, there's some challenges we consistently see in our projects and working with our customers. Everybody wants to digitally transform everybody wants to hyper automate, but they don't know where to start. And so we've had more than one project where we sell power automate, or we sell a dynamics project or something like that.
And then you get to the workshop, and they don't really know what they want to do. They know that they have a lot of need to, but they have lack of visibility into their processes. They sometimes don't really know their processes — the systems that were set up, the people left, we were working with a big bank. And they had this problem where they had these systems that were set up by people who were no longer there.
And they didn't really understand the processes. And they don't know where to start. They know they want to automate a digitally transform, but they don't know where they want to start. And kind of a kind of a mantra I have is recreating an inefficient and poorly designed process and a new system results in a poorly designed and inefficient new system.
If you don't know how you should transform it. If you don't know how the current process is inefficient. You may have documentation, but that documentation is out of date, then you have a real problem because you could wind up as bad or worse off than what you have now. And you want to make sure that you are not, you're not focusing on things that don't matter.
I'll use an example. I don't know how your garage is at your house. But in my garage, it's kind of ... I've been here several years and things pile up in it. And so I feel on Saturday, I really need to clean the garage. I get I get all motivated, I go out there. And I spend like all day on something that doesn't really matter. Like I want to rearrange the tools on the wall, rather than you really clean up the big mess if you ever had that experience.
AV: Correct. And the biggest problem here is when you do not have a plan, how to organize your garage, right? You end up throwing the most important stuff that you may have put in there for later use. Correct, right that happened to me like several times, I may have thrown in very important pieces of some parts or any electrical wiring or something like that, which I may want to use it later. So I may have thrown that away. So organization at same time plan for that is very important in my in my view, right.
JL: Yeah, that's a great example there. So this is where process mining in process advisor is pretty exciting. And a little background here. So Process Advisor was introduced at the beginning of 2020, or 2021. And the original version was what we call task mining, which is using the recorder inside a Power BI Desktop to record people doing the process. And then analyzing that, that's still there, it's still valuable. But process mining is really making the platform a lot more capable, a lot more powerful.
And essentially what that does is you have log files, right in dynamic Salesforce, SharePoint other systems, even SAP. And by taking that log data and analyzing it, it gives you real, a real AI powered X ray into your processes, and can really give you some data driven metrics that you can use to identify things such as, what are the long running parts of the process? Where is it failing? Where do we add the most rework cycles. And the something equally exciting is it's not just for one time analysis, it can be used on an ongoing near real time basis, by having a process that will refresh the data in process advisor so that you essentially can use it on an ongoing basis for a real look at your process, where it's where it's going, Well, how long it's taking, and what are those? What are those areas that should be optimized or automated? So are how do you how do you differentiate between task mining and process mining. Then, process mining and data mining, they both come under power sideways, but they come with the biggest difference.
So let's imagine if you already have data. Data can be within Azure, or data can be within Power Platform. Or it can be a third party like big lake or Snowflake or Databricks. So you want to get that into the Process Advisor. That's when you use process mining.
So imagine you already have data, you choose process mining, but when you do not have data when you want to create data to feed, whether that's when you use task mining, task mining includes recording an activity that you do as a day-to-day job. It can be Excel based actions or PDF versus actions that you do as a job. Eight to five, you use a recorder within task mining. And you feed that recording into the process sideways to get some analysis on that.
So that's the biggest difference between a process mining step and a task mining step. That's great. I think they they're not, it's not either or, because let's take a complex process like order to cash inside of your SAP system, you've got a lot of data there that can back that up. But that doesn't complete the full picture because it doesn't capture the other things that the users might be doing.
And so, what I might do is use the template for order to cash that comes for SAP as a starting point, pull that data in there, but also still have for key pieces, maybe you identify one part as long running, let's have task mining, let's record people doing the process. And then we'll get the full picture because they might be opening a spreadsheet, they might be jumping to another system, they might be going to their email to email somebody a question.
So they complete a task. That's the kind of thing that you're not going to see from your SAP audit logs. But you will see that from the task mining, so I don't view it as either or I view it as both, because we've got data, due process mining, but for things that are user intensive, do task mining.
But on the other hand, task mining doesn't see system stuff not done by users. And so, in any process, you've got both system tasks, and user tasks, process mining gets anything that's data driven, task mining gets things that actual users are doing, and they work together very nicely.
AV: That's correct. And so this is this is the high-level process, first got to have logs, gather your logs. And this is where again, for things like Dynamics Dataverse is a pretty clean, easy way to do that. Because Dataverse has very good logs, if you turn them on, you still probably may want to do some transformation, even at the basic level, like taking power query, and filtering your logs out to only include when so say you're doing field service, you probably want to analyze your work order data, you wouldn't want to feed in all the other noise, you just want to focus on that data, and then feed that in.
And this is where you can also grow this up to work with datalake storage Gen AI to Fabric Synapse, all these pieces, especially the more data you're bringing in, the more system data you're bringing in. There's a strong use case for using some of these other tools, because you got to get it down to a good log data that Process Advisor process mining can analyze.
And then you would feed that into the process mining and there's standard charts will show you just a minute. But there's a lot of standard charts and KPIs. But this is where you might want to extend that you might want to add some additional charts, they're especially industry or company specific things that matter to you. And then if you want to have this go on an ongoing basis, you can use tools like dataflow and other ETL or any other integration tools to refresh the data on an ongoing basis.
Anything you want to add to the general process there Amar. Right, so Microsoft has recently launched or enabled the feature to connect directly to Data Lake v2.
AV: Correct. So it's a very good opportunity for enterprises that store bytes size data in Datalake, right. So, you can directly connect to that data, grab the locked file, and at the same time, you can also store your process mining data into Datalake than traditionally what it is storing data was. So that's a great integration.
JL: Yeah, absolutely. And then you need to also make sure you're looking at the right things. And so there's some processes that likely are not optimal to analyze with process mining, others are more optimal. So Amar, can you fill us in on what you would look for to say, "Hey, that process might not be a good fit or Yeah, that'd would be a good fit."
AV: Right? So, I refer to one of the client examples that I've recently worked on, you know, a client they manage supply chain management right. So that when you think about supply chain management, you have to think about different things participating in a single process, it can be billing team, it can be shipping team, and it can be material CERT team, that they have a material certification team, or their own team they have for the entire process.
So, we when you think when you look at their process, right, it's completely broken — completely unstructured just because they do have different layers, different people content waiting to do a process. So, we thought, okay, we should have a process advisor in place. When we tried to do that we have, we have created multiple processes based on role perspective and team perspective, to understand process, on teams view, at the same time, roll, roll based view.
So just to get the analysis out of the entire process, break back down, break it down into multiple pieces, then and then finally, clip together to understand the whole process to identify what's the best approach that we can offer, and what's the best automation capability that we can offer to that enterprise.
So, we used, they have come up with a very unstructured process, we have, we have a process advisor in place to identify bottlenecks of every team, we broke it down into multiple pieces, then we understood the bottlenecks or automation opportunity of each team, and then we build out a very optimal process from the start till the end, to refine into the lifecycle of a process.
So, there you can imagine, when multiple teams involved in the process, right, you there are very, a lot of unknowns that you can guess, of people not showing up to the office, someone who is out of office for a certain period of time, the process lags there, if they're working on a particular case, the case lags therefore, for some time, right? So, we analyze every piece of information. If someone goes out, right, what can be the next best possibility, where the case can be picked up by a different person within the same team still contribute to the process like close that case very soon.
So, these are the things that we have analyzed. At the same time, we approached with some best practices, some good practices to determine in-house, how a case can be handled there.
JL: Yeah, that's great. And so you just summarize, if you have a process that is, you know, highly unstructured, not mission critical, done rarely and never follows the same pattern, Process Advisor may have limited value in that case, whereas if it's something that is high value, high repeatability. And especially if there's compliance or other things tied to it, there's high value in using process mining, to make sure that you are your process is not failing; to make sure you're quickly addressing it; todo root cause and time series analysis on it.
And that brings us to what are the KPIs that you get out of the box — the main ones, you know, are the process map. And that's, to me, one of the most valuable ones where I can see all the different paths that goes down, the line is thicker for the common path and then thinner for the variance and the, you know, the nice, the nice ones, and then the variance, that's going to tell me how long how many different variants of the process do we have?
You know, how many follow each one? What's the most common approach? This can be very helpful for compliance scenarios, because we can see how many times we are deviating from the process. And then there's some of them — one thing to keep in mind is to use some of some of them like the variant DNA, and to take this into your own Power BI and extend it, that does require a Power BI premium workspace.
And so, if you don't have that, that's one area where there might be some additional cost here, if you want to leverage that. That said, I feel there's strong value in just the standard metrics that you can get very quick to, stand it up and then grow it. But then if you want to use it beyond that, then or use data lakes and things like that, you can easily do this.
So this is this can be kind of the introduction, if you will, to analytics, because you can take your process that doesn't have documentation right now, analyze the data, and the tasks, come up with kind of, you know, deep metrics around KPIs around your process, and use those to make improvements. Mr. Amarnath, you would add to the KPIs or the analytical capabilities of process advisor.
AV: Oh, yes. So, so by default, when you feed in a bunch of logs, or when you feed a bunch of recordings for a process, that by default, you'll get a standard view of this overall visualization, right? But there are three different layers that you can dwell in. Until you get more insights of this process, it can be the frequency layer, the performance layer and the revote layer. The frequency layer, it creates a view that is generated based on the accounts or the numbers; or it generates a view based on the total count of actions or total count of paths in a process or frequently occurring paths.
It can be a good path or a bad path, or a path that leads, that interrupted in between. So, on the whole, it gives you an overview of frequently visited paths that have been most errors are or the disruptions just to give you insight of frequency based on the numbers.
The second view that you can also get is performance delay of view. It works with the time component, It gives you insights on actions that take most time, or the notes that that are processed quickly. So, these views give you an overview of actions that need an attention and which actions do not.
The last one is a rework, layer or rework view. So this view defines at which steps of the process, there's a loop that was created. So let's imagine you send and wait for an approval to happen. So that's, that's one of the steps can in a process. At times, approval can be rejections, too. So you have to rework on something and send it for an approval. So that's potentially a loop. So this, iOS gives you a loop occurrence. And it gives you some insights on how to mitigate such loops. So these are different types of views that you can go ahead and create different views on top of the standard views that you traditionally get.
JL: That's great. And the real, what we're shooting for here, is real business justification for digital transformation, and automation. And this is something that without a tool like process mining, it's a hard question to answer, we think we know what it is, we think we know what the justification what the real return on investment is going to be. But we don't know for sure where the data doesn't lie. And so we can from this get real insight on to how long manual tasks that people are doing with product with task mining.
And from data in your system processes, how much time is being taken, how much failure there is how much rework there is and use that to actually calculate a strong ROI. Because if we see, this part of the process is taking five hours. And we could automate that. We had one customer, we took a six-hour process to a 58-minute process, that is a huge return on investment from that.
And so, this will help you then make sure like when you're cleaning your garage, that you're focusing on the highest value things first, now that you don't want to do the other things but get the big rocks first. And then for the things you're not automating how should we digitally transform and optimize them, if we see that a lot of people are varying from the path or one person's more optimal, and the way they do it, let's train them and upskill them and how to how to do that.
And basically, use this as an ongoing change driver within our organization. So, we're gonna have to wrap it up soon here. But we like to end with our three thoughts here. So what are three takeaways, Amar, that you would say that people should keep in mind? We might we can have more than three. But you know, three is a good number. So, what do you think?
AV: Right? The first thing is, as always start small, like if you have a process advisor, and every other licensing handy, just go ahead and start with what you already have as a data. You can connect to Azure, you can connect to your DevOps instance, as well to look at the work items and how and how they're transforming into decision making things.
Number two, if you have four platform deployments already, you can connect to a power app, you can connect to flow on the RPA to get the insights of the Flow runs or the app telemetry.
JL: That's a great point and isn't there there's now a button inside of the flow studio that says, "analyze this flow with process with process mining," where basically from the flow itself, it will look at the logs for your flow and show you where it's how many times it's failing. Other things like that. That's a great one. And then what was your third item for a before I rudely cut you off there, Amar?
AV: Oh, yeah. So the last one that I would imagine is after working on post service for a while now, so you may, you may, you know, completely get, get lost focus on the data looking at the data, right. So always think about a single from a single perspective. Now you can create a number of processes, combine them into one at the end of the day, and still visualize that as an entire process. But let's say you like I said before, if you have different teams participating in a single process, always create process based on a team. So, a team's point of view, it will be much focus than overall point of view.
JL: Yes, those great one, so I'm gonna give you my three, so we each get three. So my three are, you've already got it, you somebody in your company likely has a Power Automate license, and you can start using it now without investing any more money.
So start small, as Amar said. Take in some from your Power Apps from your Power Automate flows; from your Salesforce — from whatever system. You have logs, you know, let's do a pilot of it and get it going and get comfortable with it and see what insights we can get we can bring in.
Secondly, you can extend it, and you can have it run on a continuous basis. So, it's not just a one-time analysis tool. It's an ongoing analytical tool to help you continually make improvements.
And thirdly, tied to number two is Hitachi Solutions is uniquely positioned as a partner to help you with us, we work closely with Microsoft on process mining. We bring not only, you know, a top-notch Power Platform and automation team, but we have a data and analytics team that can help you with your logs, as well as extending the visuals in Power BI.
We have a business advisory services group that is very, very good at what they do. Please keep in mind, process mining and process advisor will help you identify where your current processes inefficient, it won't necessarily tell you how your new process should be.
And that's where business advisory — especially for complex processes — is critical just to make sure we're designing something that will really meet the challenges; meet and address the problems that you're having. So again, process mining plus automation, you know, is good, but the missing piece there. And the thing that really kind of brings it home is the business advisory services to help you identify what you know, again, now that we know what's the problem with your current process.
And we want to automate it, we want to optimize it, how should we do that? And we can help you with all those areas. So even starting small or growing it for even bigger, more complex processes.
So thank you for listening to our podcast today. Thank you Amar for your time and joining us and sharing your great experience with this. Remember to subscribe to Exchanges with Hitachi Solutions wherever you get your favorite podcasts. You can find our entire episode library at our website: global.hitachi-solutions.com
###