The ability to measure the effectiveness of your Knowledge Management (KM) program and the initiatives that are essential to its success has been a challenge for all organizations executing a KM Program. Capturing the appropriate metrics are essential to measuring the right aspects of your KM Program. The right metrics will facilitate a clear and correct communication of the health of the KM program to your organization’s leadership. In this post, I will identify metrics (or measurements) of four key initiatives of most KM Programs. These initiatives are: Communities of Practice, Search, Lessons Learned, and Knowledge Continuity.

Community of Practice (CoP) Metrics

Typical CoP metrics include:

Average posts per day, Unique contributors (People posting at least once), Repeat contributors (People posting more than once) and Majority contributors (Min people for > 50% of posts).

Some points to consider:

• Recognize the diversity of interests in those participating in the group, and that this is a voluntary undertaking for all involved.
• Develop a stakeholder classification and perform a RACI assessment for each stakeholder group.
• Through a collaborative process, arrive at coherent goals, objectives, principles and strategies for the group.
• Develop a CoP plan with agreed upon moderator criteria and stakeholders that influence group behavior in ways that are congruent with the group’s goals and objectives.

Search Metrics

Search Metrics are determined through Tuning and Optimization

Site Owners/Administrators should constantly observe and evaluate effectiveness of search results. Site Administrators/Owners should be able to gather Search Results reports from the KMS administrator periodically (every two weeks). From these reports, they can analyze the type of keywords users are searching for and from which sites most of the search queries come from. Based on this, Site Administrators/Owners can add ‘synonyms’ for their sites. If any newly added metadata column needs to be available in Advanced Search filters then the request must be sent to the KMS administrator.

Search Metrics

• Search engine usage – Search engine logs can be analyzed to produce a range of simple reports, showing usage, and a breakdown of search terms.
• Number of Searches performed (within own area and across areas)
• Number of highly rated searches performed
• User rankings – This involves asking the readers themselves to rate the relevance and quality of the information being presented. Subject matter experts or other reviewers can directly assess the quality of material on the KM platform.
• Information currency – This is a measure of how up-to-date the information stored within the system is. The importance of this measure will depend on the nature of the information being published, and how it is used. The great way to track this is using metadata such as publishing and review dates. By using this, automated reports showing a number of specific measures can be generated:

1. Average age of pages
2. Number of pages older than a specific age
3. Number of pages past their review date
4. Lists of pages due to be reviewed
5. Pages to be reviewed, broken down by content owner or business group

User feedback – A feedback mechanism is a clear way to indicate if staff is using the knowledge. Alternatively, while many feedback messages may indicate poor quality information, it does indicate strong staff use. It also shows they have sufficient trust in the system to commit the time needed to send in feedback

Lessons Learned Metrics

Lessons Learned Basic Process: Identify – Document – Analyze – Store – Retrieve

Metrics are determined and organized by key fields from the lessons learned template and includes responses gathered during the session. Lessons Learned should be identified by Type of lesson learned captured (i.e., resource, time, budget, system, content, etc.). Summarize the lesson learned by creating a brief summary of the findings and providing recommendations for correcting the findings (i.e., Findings – a summary of the issues found during the review process; Recommendations – recommended actions to be taken to correct findings). In order to provide accurate metrics the approved actions should be documented and tracked to completion. In some cases the approved action may become a project due to high level of resources required to address the finding. Some metrics include: Impact Analysis (time (increased/decreased), improper resourced, budget constraints, software/system limitations, lack of available content, etc.); Applying lesson learned: % of Problem/Issue solved with lesson learned per category and overall.

Knowledge Continuity

The keys at the heart of knowledge continuity include:

• What constitutes mission-critical knowledge that should be preserved?
• Where is the targeted mission-critical knowledge and is accessible and transferable?
• What decisions and action are required to stem the loss of valuable and in many cases irreplaceable knowledge?
• Successfully obtaining, transferring, and storing the lessons learned and best practices from their most experienced and valuable workers to a knowledge-base or (KM Application) before employees depart or retire?

Some Metrics Include:

• Percentage of knowledge harvested and stored from key employees.
• Percentage of knowledge transferred to successor employees.
• Cost associated with preventing corporate mission-critical knowledge from loss
• Provides a structured framework and system to store, update, access, enrich, and transfer to employees to support their work activities
• The amount of ramp-up time of new hires, moving them rapidly up their learning curves and making them more productive sooner

Let me know if you agree with the metrics identified here and/or if you know of additional metrics within these key initiatives that must be captured. I look forward to your responses.

About the Author:  

Dr. Anthony J. Rhem leads the KM Institute's "Information Architecture and ECM" training and certification program (next class - March 21-22, 2018).  He serves as the President of A.J. Rhem & Associates, Inc., a privately held Information Systems Integration and Training firm located in Chicago, Illinois. Dr. Rhem is an Information Systems professional with over thirty (30) years of experience, a published author, and educator, presenting the application and theory of Software Engineering Methodologies, Knowledge Management, and Artificial Intelligence.

In addition Dr. Rhem serves as a Professor of Knowledge Management and Director of Research at The Knowledge Systems Institute - Master of Science Knowledge Management Program.