ERP Utilization Series: Measuring User Adoption and Engagement

It is generally accepted that user adoption and user engagement are key critical success factors for ERP business value realization.  However, a majority of ERP implementations take a cursory approach where a qualitative method was utilized for measuring user adoption and user engagement.  A qualitative approach focuses on data that is subjective, interpretive and exploratory.  Let’s see how successful this soft approach has been in the past.

According to a 2014 survey from Panorama Consulting Solutions of 192 businesses, 66 percent of respondents reported receiving 50 percent or less measurable benefits from their ERP implementation, and less than 40 percent were satisfied with the rate of user adoption. In the US, a recent study estimates that over half of all Accounting, ERP, and CRM implementations fail to meet their objectives. In Europe, the corresponding statistic is 70%–80 %!  An oldie but goodie is what was learned from the 1999 Hershey’s ERP failure (user adoption).  A recurring theme from the ERP failures listed above is a lack of actively tracking and managing user adoption and user engagement.  Traditional approaches of End User Testing (EUT), Just-In-Time (JIT) training and user surveys are insufficient for a reliable, effective user adoption and engagement process.   In the following sections, we will discuss practical steps that you can take to better quantify user adoption and user engagement.

User Adoption vs. User Engagement

The first step to take in implementing a measured and repeatable approach is to understand the difference between user adoption and user engagement.  User adoption is when users adopt a system that works to fill a specific need.  User engagement measures how effectively a system is utilized.  A user can adopt an ERP solution but yet the user is not making the most of the ERP functionality available.  In my humble opinion, I believe that we in the ERP industry too often try to broadly address these specific areas with qualitative surveys which are translated into quantitative data with some level of bias. 

Don’t get me wrong, using surveys and statistics as a tool to provide some level of quantification of qualified user feedback is an established approach.  The problem occurs when user surveys are the ONLY method we use to manage user adoption and user engagement.  Surveys were necessary in what I call the 1^st generation of ERP given that the technology and availability of user data was not a primary objective for ERP vendors.  However, now that we are in the 2^nd generation of ERP, user data is more available for consumption.  We in the ERP industry have to leverage this new capability in addition to conventional means.

What data should we go after to highlight user adoption and user engagement?  Consider the following illustration as a starting point.

Starting with user adoption metrics:

1. Test cases executed:  Quantify how many end-user test cases are executed by ERP users.  There is no better method for training and fostering user adoption than “hands-on” experience.  Automating testing may not always be the best option for all scenarios.
2. Number of users:  Quantify how many users are provisioned for the ERP solution.  An increase in ERP users could be an indicator that organizations see the ERP solution as an enabler.  Conversely, a decrease in ERP users may highlight a risk.
3. Consumed training:  There are three attributes to capture in the area of training: (1) Total # of hours consumed, (2) Total $ spent on training and (3) Average training hours per user.  An “intuitive” user interface is not a substitute for user education and training.
4. Number of ERP sessions by connection type:  Another perspective of measuring ERP user adoption includes (1) number of user sessions by connection type (online, mobile). Mobility can be a huge enabler for greater data visibility and accelerating approvals.  Capturing this data will provide customers deeper insight regarding how their users consume their ERP service.

Continuing with user engagement metrics:

1. Number of manual transactions:  “A major overhead in operating ERP systems is entering transactions. Transactions take time, cost money and introduce the possibility of errors.” Making ERP Work, Sam Graham.  If you are interested in process efficiency and ERP business value realization then the customer should take a hard look at these scenarios and have a plan to minimize these activities.
2. Number of correcting transactions:  Correcting transactions are rework of existing transactions.  If you are a student of Lean Six Sigma like I am, then customers want to do everything they can to minimize rework.    This type of rework may seem insignificant at the individual user level, yet the customer will be surprised when (1) the rework summarized across the organization and (2) the customer quantifies the impact(s) to business process cycle time(s).
3. Number of exception transactions:  Any competent ERP solution allows some level of exception transactions.  Example: At month end, the user makes an Accounts Payable journal entry in the General Ledger and then the ERP solution automatically accounts for the journal entry in the Accounts Payable sub ledger.  Another example of an exception transaction is when a transaction is rejected (Expense Report).  Exception transactions are a necessary evil.  However, it is important to note that exception transactions generally require (1) additional user effort (costs) and (2) slows down the business process cycle.  Tracking this type of data will provide insight regarding the root cause for cycle time exceptions.
4. Decision support transactions:  A common value proposition that all ERP vendors are making today is enabling users to have more time to focus on making business decisions by automating tactical business transactions and consolidating data into meaningful information.  Therefore, we should be able to measure this effect by categorizing and capturing this information via the ERP solution. For example, if we observer that the number of manual transactions are decreasing and the number of decision support transactions increasing then we can feel confident that the organization is effective in utilizing the ERP solution. 

Following is an example of an ERP solution user engagement model.

Let’s elaborate on a couple of points.  First, user adoption does not equate to effective user engagement.  The majority of ERP solutions provide the customer the ability to perform transactions efficiently (automated) and inefficiently (manual, exception, corrections).  As a customer, part of your responsibility is to identify inefficient use of the ERP solution and create an improvement plan.  From the above profile, we can identify the following:

• There is an unhealthy volume of manual transactions.  As the volume of manual transactions increase, it is reasonable to conclude that there will be a greater increase in exception and correction transactions (resulting in additional cost/cycle time). 
• It is possible that the increase in manual transactions is due to a limited configuration of the ERP solution and the user’s knowledge of the ERP’s features.  Configuration issues can be identified by the customer’s support team and the ERP vendor’s support resolutions.

As the customer performs user enablement analysis, it is important to note that a visible pain may be a consequential result of an underlying issue.

Practice Makes Perfect

User efficiency with an ERP service is not a Just-In-Time event but an incremental process.  Given that people are the most important component of a business solution, it is negligent not to involve users thought out the ERP service implementation.  Sorry for the tough love but this is a gap that no one wants to address head-on.  It has been identified as a critical success factor but is typically handled as a “negotiation play” during the sales process.  If a customer is smart, then they can use data like user adoption and user engagement as leverage with the ERP vendor and SI partner. 

Incremental user involvement during the ERP solution implementation requires greater project management coordination and guidance.  Second only to the ERP vendor, end users directly impact business value realization for a customer. 

Rising to the Challenge of Creating Reliable User Adoption and Enablement

Referring back to my previous article, effectively business value realization will require an investment from the following stakeholders.

Challenge to SI Partners

• User Adoption and User Engagement should be generally accepted as key project management metrics (On-time, On-budget, In-Scope).
• SI Partners should deliver and execute a knowledge transfer plan for each ERP implementation.
• SI Partners have an incremental user involvement during the ERP solution implementation. 

Challenge to Customer Leaders

• Customer should make an investment of at a least 20% investment in user training and user enablement.  Consider the following guidance:
  • IDC Learning Services Research shows that organizations, on average, spent 15% of their total ERP budgets on training, while indicating that a 17-20% investment yields better results.
  • Gartner Group research shows similar averages and an interesting fact: organizations that spent less than 13% of their total ERP cost on training were three times more likely to see their ERP projects exceed deadlines and run over budget when compared to organizations that spent 17% or more.
  • Aberdeen Group says that ERP is not a “set it and forget it” tool. Implementing a continuous learning strategy will help users complete their tasks successfully pre- and post-launch.

Challenge to ERP Vendor

• ERP vendors should deliver out of the box user adoption reporting and user engagement reporting as described above to enable customers in actively managing ERP service utilization.
• ERP vendors should provide support service analyses that are a result of incorrect configuration or lack of training. 

Challenge to Business Users

• Business users must be open to change and innovation. Sticking to the legacy process and keeping things the way they are at all costs reduces the chance of real optimization with the new ERP solution.
• Users must broaden their horizon from a functional job focus to business process focus.  This requirement is not just for super users.  Every individual ERP user should have an understanding of how their input impacts the overall business result(s).   

Summary

“Without data, you’re just another person with an opinion.” E. W. Edwards Deming.

For far too long, ERP user adoption and engagement management has been strictly a qualitative exercise given that there has been limited services provided to easily quantify user metrics.  One type of data is objective, to-the-point, and conclusive. The other type of data is subjective, interpretive, and exploratory.  Given how important user adoption and engagement are to ERP solution success, there should be a comprehensive approach including both qualitative and quantitative data for ERP user involvement.

Join the community! 16k followers across 100 countries!