Success Measures 2: This is part two of our two-part post on Success Factors and Measures. Two independent events last month (a magazine interview and a webinar) resonated around a frequently-discussed, but often disputed topic: What is project success, and how do you achieve it? The events covered two aspects of project success: the Success Factors (that lead to project success) and the Success Measures (used to evaluate success). This posting covers the Success Measures.
The Success Measures
Tim Jaques and Frank Salidis ran the latest webinar in the IPMA-USA 2010 Dialogue series the first week of July. The topic was Perspectives on Project Success: Excellence in Project Management. The well-presented and discussed Dialogue was excellent, but there is much more to the topic than an hour’s time. Some of the key points included the fact that the Triple Constraint is merely a project measure. It is certainly not as important to the end-user as such hard-to-measure items as business results and customer satisfaction.
Other points included discussions about tangible and intangible value, including Return On Investment, and Stakeholder satisfaction. Perceived failures, at least according to project measures, may be successes by the time of product success measurement. A key example provided was the Sydney Opera House. The distinction made: Project outputs versus project outcomes.
The webinar introduced this complex topic; the topic deserves a lot more discussion. After all, how can one hour-long Dialogue cure the years of neglect we’ve seen in measuring organization benefits from projects? Let us explore this topic a bit further, organized around several key points:
- The Most Important Measures
- Responsibility for Measures
- Reluctance to Measure
- Those Difficult Intangibles
- Timing of Measurement
Success Measures 2: The Most Important Measures
The most important project success measures are those factors that are important to the business or organization unit. Some organizations use the term Business Driver to focus the project on business results. One set of universal Business Drivers comes not from project management, but from Strategic Planning. Drawing from that discipline, the purpose of a problem-solving or opportunity-seizing project (select one) is to:
For a Problem-Solving Project
For an Opportunity-Seizing Project
The Business Driver expresses the Why of the project. It is either embedded in the first five words of a single Objective, or used in conjunction with a series of What-oriented Objectives. Experience has shown that a project that has multiple business drivers will tend to fail to meet them all. Thus, one business driver must be the focus of the project. Others may be “realized if possible” (the happy accident), but you should never try to achieve multiples. Note that we have used the above universal list since our Strategic Planning days of the early 1980s. But only recently has “Meet regulatory requirements” become such a frequently-used business driver.
Success Measures 2: Responsibility for Measures
Who is responsible for establishing success measures, and then following up on them? It must be (internal) line of business Customer Managers, in the groups the project affects, for several reasons. First, they are in a position to evaluate and measure the project results in the organization. Secondly, the project team is usually long gone by the time of the final benefits measurements. A less obvious reason is this: Actions of internal Customer Managers have direct impact on whether the project is successful.
Even with Competent and Performing project managers and teams, stretched-thin internal Managers may fail to deliver. This happens when they cannot afford to provide access to the right customer staff during the essential engagement periods. Those periods include requirements elicitation, test planning and implementation, user documentation and training. Post-project organizational change management is yet another essential period. Failure to include this engagement dooms the project from the start. Do you think the Manager who could have saved the doomed project is eager to measure the results? No, they were already frustrated because, despite their best intentions, their hands were tied.
In all our project methods, we identify a key role for all projects, the Customer Representative. This is a Manager from the business area, who could speak for the project’s Customers. We cite managing Time and Cost as the Project Manager’s responsibility, and managing Scope and Quality as the Customer Representative’s responsibility. Both share responsibility for managing Risks and Talent. The rationale: Even a competent project manager has difficulty balancing all the (as we called them) Vital Signs. As we’ve written in our popular Project Levers And Gauges article: successful project teams manage both the trailing indicators, and the leading ones. This was one way we assure that both groups of indicators succeed.
Success Measures 2: Reluctance to Measure
There are many reasons for a reluctance to measure success. Organizations that justify projects using Cost-Benefit Analysis, or Return on Investment, resist showing how many staff positions were “saved”. This unfortunate measure is usually evaluated by counting the heads as they roll out the door. Few organizations have the internal measurements in place to show how all those “heads” were immediately re-purposed. Rather than reduced head-count, those staff moved into other essential-but-resource-starved business functions. Too bad that cutting headcount is the only obvious measure to so many. A significant business training opportunity would be to show middle Managers how to value and verify business benefits from projects. That training would include coverage of the intangible or hard-to-quantify benefits.
Those Difficult Intangibles
Those Intangibles are difficult! Some of them are multi-variant, and may require alignment of all the planets in the solar system for them to come true. Others are multi-responsibility: No one Manager has control or even influence over all the variables needed to achieve them. Yet, we have seen $100M projects justified mostly by intangibles, And that is even outside Information Technology projects, where no one can agree what “user friendly” means. And, there we may have problems finding friendly users in the first place.
Timing of Measurement
Some organizations wait until long after the project is completed to measure its success, which makes some sense. After all, you cannot measure success until you’ve had some! And besides, maybe if we ignore measuring our results, Executive Management might forget that we promised this new product that the project launched will grow quarterly earnings 25% per quarter within 2 years after completion. Pretty embarrassing that the market changed significantly just before launch. Maybe it’s a good thing CEOs change every three years in your industry.
Effective organizations begin evaluating the success measures at project initiation. After all, they need to know the baseline measures. Before Requirements are complete is a smart time to update the Success Measures that were originally stated at Portfolio Prioritization. They may now be obsolete. And the most effective organizations trace the indicators that will result in final success throughout the project. That includes each Milestone or Stage Gate approval and while evaluating each requested change. This success-focus also establishes heightened Management vigilance for the out-of-control projects. These show lack of clear status for time-cost-quality-talent-risk-scope measures, that we’ve too often seen in the last third of some projects.
Success Measures 2: To Summarize
We said above that the excellent Success Dialogue only began the true discussion about Measuring Project Success. And our comments above may merely add more questions. It is up to you, dear reader, to add your perspective to this discussion. What is Project Success, from the Executive’s or Manager’s point of view? From the end-user’s view? From the Project Team’s view? How do you measure success? When do you measure it? Who is responsible? And, who has a success story you can share, about demonstrating and measuring project success?