What if your organization is in chaos but your metrics don’t show it? You measure departmental or organizational successes in a textbook manner using EBITDA, revenue growth, and efficiency ratios. But hiding underneath the wallpaper is something scarier than you ever imagined. The goal is to send those Key Performance Indicators (KPIs) up and to the right. Yet in immature organizations, success is the result of individual heroism, not scalable systems. A few people performing superhuman efforts pull the weight for the whole team, but no one understands how or why.
Then you start implementing a new process, and immediately metrics plummet!
- The deal desk slashes your pipeline!
- The new testing framework sends defect levels sky-high!
- The productivity tool you rolled out slows your teamdown as they learn to use it!
Is everything broken?
No, your metrics just aren’t keeping up with reality. Numbers don’t lie, but they can mislead. As organizations mature, they evolve their KPIs. The metrics that measure a well-run system won’t fix a broken one. A well-oiled machine needs different KPIs than a department in chaos. Organizations using the wrong KPIs often misinterpret early results and make poor strategic decisions.
Consider these parallels:
Parenting a Toddler: The first time you set rules, it feels like all you get is tantrums. If your KPI is “obedience rate,” early results scream failure. New rules trigger pushback. Over time, those tantrums subside. What emerges is structure and discipline that set the foundation for growth, and maybe even a well-adjusted adult.
Renovating a House: Upgrading often starts by knocking down walls and tearing up floors. It looks like destruction, but it’s the first step toward a dream home. If your KPI is “usable square footage,” you start off in the opposite direction, losing space before you gain it.
Organizations introducing structured processes go through the same messy, frustrating phase before things improve. When you introduce a structured process where there was none, it often feels like things are getting worse before they get better. That’s why setting the right KPIs at the right stage is crucial.
The Challenge of Initial Metrics
When an organization first introduces structure to a previously chaotic process, performance can decline rather than improve. This is because the first iteration of structured evaluation often reveals previously hidden inefficiencies and problems. Understanding this dynamic is critical to setting the right KPIs at the right time.
Here are a couple of examples to illustrate the key concepts:
Example: Software Testing
Consider a company just beginning structured software testing. Before formalized testing, few defects are identified, not because the software is high quality, but because no one is looking. As structured testing begins, defect counts spike. This can create the false impression that the new testing process is failing and that suddenly the product is buggy. In reality, the process is exposing long-hidden issues.
Over time, defect prevention improves and testing matures. At that stage, more advanced KPIs such as defect resolution time or the percentage of automated test coverage become better measures of effectiveness.
Example: Deal Desk Implementation
A similar dynamic occurs when introducing a deal desk process to qualify and review sales opportunities. Initially, the deal desk may reject quite a few deals, causing a perceived drop in pipeline value. However, this early stage filters out low-quality opportunities, ensuring that only viable deals remain.
If an organization only looks at “total pipeline value” as a KPI, it might conclude that the deal desk is harming the business. In reality, it’s just cutting out the junk. Instead, they should focus on more appropriate early-stage KPIs. These may include the percentage of deals that advance past initial qualification or the average deal size after implementing the process. As the deal desk matures, more refined KPIs such as deal cycle time or win rate will provide better insights into its long-term effectiveness.
Understanding the Capability Maturity Model (CMM)
To effectively adjust KPIs, leverage the Capability Maturity Model (CMM), a framework that categorizes process maturity into distinct levels:
Adjusting KPIs as Maturity Increases
Organizations must align their KPIs with the Capability Maturity Model (CMM) level of the respective system or process being improved. At lower maturity levels, KPIs should focus on identifying inefficiencies, establishing consistency, and process compliance. As processes stabilize and improve, KPIs should shift to measure optimization and effectiveness.
For example:
- Chaotic/Initial Stage: KPIs focus on process establishment and identifying problem areas (e.g., defect density, rejected deal percentage).
- Managed Stage: KPIs assess consistency and process adherence (e.g., percentage of tested code, percentage of deals reviewed).
- Defined Stage and Beyond: KPIs evaluate effectiveness and optimization (e.g., defect prevention rate, deal closure rate, customer satisfaction post-sale).
The Danger of Premature Optimization
One of the most common mistakes organizations make is using advanced KPIs too early. The danger comes from evaluating a process based on defect prevention rather than defect detection. An organization may falsely conclude that its early testing efforts are failing. Similarly, if an organization judges its first deal desk by revenue impact rather than pipeline quality, it may abandon a successful process too early.
Aligning KPIs with Organizational Maturity
Organizations must recognize that initial implementation phases will often expose inefficiencies. Early KPIs should focus on surfacing and addressing these issues. As maturity increases, more sophisticated metrics can take their place. By aligning KPIs with organizational maturity, leaders can avoid misinterpretations and make data-driven decisions that support long-term growth. The key is knowing what to measure when.
