Click the items below to see answers to a variety of questions related to the Asuret implementation risk assessment service.

    How big is the problem?
    What is the root cause of most project failures?
    What is the anatomy of project failure?
    What events serve to shatter denial of project failure? 
    Upon project failure, should you kill the project?
    Upon project failure, should you reset the project?

It is estimated that of the $255 billion spent per year on information technology projects in the United States, more than a quarter is lost to failures and cost overruns. The Chaos Report by the Standish Group, considered a landmark study, looked at the causes of software development failure. Their sample included large, medium, and small companies across major industry segments: banking, securities, manufacturing, retail, wholesale, heath care, insurance, services, and local, state, and federal organizations. The total sample size was 365 respondents representing 8,380 applications.

The report shows that:

• 31.1% of IT projects will be cancelled before completion
• 52.7% of projects will overrun their original cost estimate by 189%
• US companies will spend $81 billion for cancelled IT projects
• They will pay an additional $59 billion for ongoing IT projects that exceed their time limits
• Only 16.2% of IT projects are completed on time and on budget (in larger companies this rate drops to 9%, and only 42% of these retain the original features)

More recent studies do not report findings that are much more comforting. Three academics from Oxford and Simon Fraser Universities, Chris Sauer, Andrew Gemino, and Blaize Horner Reich, describe their results in an article titled The Impact of Size and Volatility on IT Project Performance, in which they claim that one third of all IT projects fail. While this is a lower estimate than that posed by the Standish Group, it is still disheartening. Despite improvements in software and IT processes, a generous estimate is that IT project failure rates remain plateaued at 30%. For American and European companies, the lost opportunity costs associated with these failures and overruns could easily be extreme.

These are dismal numbers with potentially calamitous consequences. While those involved in these scenarios have the best intentions, without an understanding of the true reasons for failure, they are likely to experience a development route that is fraught with expensive miscommunications, misalignments, and misery.
 

IT project failures emerge from the void that exists between the business and the IT organization. Historically, in the days of early computing with large rooms filled with whirring and clicking mainframes, IT was a highly specialized field that involved sensitive equipment. Software and hardware professionals were the machines’ protectors, keeping uninformed users at arm’s length to preserve expensive computing instruments. IT people were also the priests, servicing the computers and acting as intermediaries between machines and users. And they were the interpreters, translating technical capabilities to business strategists who were at the mercy of those who managed the machines. They even developed a unique and impenetrable vernacular filled with highly technical terms and acronyms. A natural divide therefore arose between IT and the business, right from the origins of computing.

As technology grew more democratic—due to the advent of personal computers, desktop applications, the Internet, social software, and cloud computing—individual users could enter the sanctum. They could at times work outside the firewall and even modify applications. However the legacy of IT as a group apart from the business lingers. With the proliferation of technology, IT is no longer seen as the priesthood. IT is now a service provider. A true strategic partnership between IT and the business, in most circumstances, is absent.

This divide then is the shaky ground on which IT projects are often conceived. The business side focuses on strategic possibilities and long term, far reaching transformations; while the IT side lives with technical complexity, detail, and precision, often disconnected from the overarching business goals. In fact, the business may make its technology demands based on business realities and predictions, without a full understanding of IT capabilities, limits, budgets, and processes; while IT may develop a system based on technical and implementation realities that, of necessity, may truncate or eliminate some business requirements or exceed budget and time restrictions. While the two sides are committed to success, they are pointing in different directions.

As Ed Yourdon (noted author and respected software methodologies and project failures guru) said in an interview with Michael Krigsman, ZDNet blogger and President of Asuret:

The question is: whether IT is building the kinds of systems the business needs, or whether they anticipate, and can work strategically, to help the business make the best possible use of IT. That problem has been around for 30 or 40 years.

Given this historical background, should we be surprised that so many IT projects fail?
 

Against the backdrop of IT failure's history, the scenario that births many new IT projects is, itself, flawed. Often the management team, in top-down isolation, discusses the business need and from that defines the IT project—its scope, budget and timeframe. The project manager is then brought in to receive this definition of requirements and resources and is told to deliver. The project manager conveys the business requirements to her team, which dissects them and discovers that these requirements, as defined by leadership, are incompatible with the authorized budget and time goals. Management has unknowingly built failure directly into the project’s foundation.

The project manager reports this news to the management team. If management adjusts the original plan, based on the project manager’s feedback, the descent towards failure can be halted. However, if management remains committed to the established requirements, budget, and time frame, and pressures the project manager to deliver the original scope, the project will likely fail to meet these expectations. The project manager, then, feeling disempowered and somewhat threatened, returns to her troops and demands of them what has been demanded of her – to perform as if success were possible. The initial conditions have put the project on a trajectory for misinformation, misalignment, and failure.

Andrew Shimberg, senior consultant for nGenera, calls this phenomenon “fact-free planning.” Leadership establishes unworkable goals and requirements and then ignores advice to the contrary from the experienced project team. Fact-free planning sets the stage for repeated cycles of flawed conversations, false commitments, padded expectations, and conflict avoidance.

What makes it so difficult for an organization to address a failing IT project more aggressively? In some organizations the culture is an obstacle. A conflict-avoidant culture that tends to shoot its messengers will not invite the kinds of cross-boundary, clarifying conversations needed to address IT project challenges. Another reason that these conversations are hard to have is that companies tend to treat IT projects differently than non-technical business initiatives. Companies may tend to be either deferential or disrespectful toward IT and behave as if IT projects had their own set of rules. IT projects do have their differences in that they are technical and the people involved have a unique set of skills not shared by business people; and IT projects often require negotiating complex relationships among software vendors and system integrators (a technical arena). IT, however, may consciously or unconsciously use these differences to shield itself and maintain control over the project, while the business may, consciously or unconsciously, use these differences to maintain a distance from IT. With each group, hunkered in its own comfort zone, the gap widens, and problems, couched in a language and set of processes foreign to the other group, continue unaddressed.

In addition, IT departments may not be given appropriate levels of respect from the business that sees them as service providers not partners. And the IT department may see itself purely as developer and maintainer of the corporate infrastructure and may not know how or see the need to become more of a partner with the business. Each group remains in its own silo, and this separation becomes a persistent reality for many organizations.
 

Failures are multi-causal, develop over time, and progressively reveal indicators of breakdown with increasing frequency and urgency. These indicators usually first emerge from inside the project team, the group that is closest to tracking scope, budget, and schedule. The team may attempt to alert management; or, given the pressure to deliver, the team may continue to offer the appearance that everything is still on track. If the team does communicate these early indicators to management, the leaders may respond wisely and prevent project failure; or, out of their own sense of urgency to deliver, they may deny or distort the evidence of failure. As we have seen, statistics show that the path towards failure often pushes past this attempt at correction, and management often tends to ignore or misinterpret early indicators of impending negative outcome.

At some point in the life of the project, this denial is shattered by a decisive moment of truth – when the gap between the intended goals and progress towards those goals becomes blatantly evident, and failure can no longer be denied. When this moment arrives, neither management nor the project team can avoid confronting the issues disrupting the project.

There are typically three precipitating events that serve to shatter the denial:

1. An intermediate or late stage project review is held during which management begins to talk about serious issues such as going live, training users, or cutting over from the a legacy system. The gap between perception and reality gets splayed across the conference table for everyone to see.
2. The project has consumed, or is about to consume, its budget and has extended beyond its allotted time, and the project manager is forced to ask for additional, unplanned resources.
3. Serious project issues and impact have become visible to external stakeholders such as customers, vendors, or partners.

As a result, continued denial becomes impossible.

Converting this moment of truth into a healthy outcome requires great self-awareness, discipline, flexibility, and courage. Many organizations, respond in the form of continued denial of root causes accompanied by blame and scapegoating. The management team sincerely wants the project to succeed and the company to prosper, but in the wake of looming failure and its personal and organizational repercussions, they may focus on quick fixes and scapegoating accessible targets, such as the project manager. Rather than analyzing the situation objectively through open discussions and information transparency, managers in this situation often make reactive decisions with questionable benefit and little long term-value—decisions that do not substantively improve the project.

Management may point its collective shaking finger at the project manager. They may order her to find a way to make it right, or they may find a replacement. The existing, or the new, project manager, reprimanded and fearful of losing her job, fiercely passes management’s orders to the project teams. She may require them to work more hours and on weekends and to refrain from using vacation days. The teams do their best; but since nothing has really changed, except for the pressure, the obstacles that interfered with success before do so again.

Management may throw more money or more time at the project. In some instances this can permit a derailed project to undergo the changes it needs in order to make corrections and deliver value. However, without careful attention being paid to the underlying causes of the problems, more resources and time may just permit the distortions to continue, which makes the project, whether it ultimately succeeds or fails, more costly.

The cycles of denial and blind hope, of quick fixes and blame, can continue, unabated, for extended periods. As long as the symptoms, the visible manifestations of failure, remain the focus of interventions instead of the true causes, this pattern will continue. There are two likely conclusions: the project fails entirely or restarts as a new project with a new plan. Few projects locked in this pattern succeed and achieve their initial goals.
 

A key reason to kill a project is that it no longer serves a meaningful business imperative, and therefore the value no longer outweighs the risks and costs. Either because of project delay or a change in the business environment, the project’s intended objectives have become obsolete or are no longer worth the investment. A second reason to kill the project is that the organization does not possess the ability to execute the plan because internal or external resources are inadequate, unavailable, or inaccessible. In extreme cases, a third reason to kill a project is that the project has fatally compromised the organization’s financial stability or its operations. American LaFrance, a leading manufacturer of firefighting vehicles, declared bankruptcy and blamed a failed ERP implementation as one of the causes.

Once the decision to kill has been made, it should be implemented decisively. Delays or ambiguous commitment allow for the reemergence of more denial-powered attempts to reanimate the project. The slide back to “fact-free planning” (when leaders establish unachievable goals and then ignore feedback to the contrary from the project team) is always looming.

It is important to remember the troops when making a kill decision. They have been dedicated to the project for many months, possibly years. By now, they identify with the project and have invested their intelligence and emotion into making it successful. The kill decision must be made with great consideration for the people who have been working very diligently to keep the project afloat. Regardless of the destiny of the project, these people remain critical intellectual resources for the organization. Their morale and loyalty must be sustained.
 

The decision to reset rather than kill a project is made because a strategic, financial, and operational evaluation shows that the project, with modifications to scope or execution, still offers defined business value. The reset decision is almost as important as the initial decision to launch the project and must go through similar rigorous initiation steps. However, the reset decision can benefit from the experience from the failed first attempt. With lessons learned in hand, the reset project should avoid the problems that plagued the original project and achieve success.

One critical reset success factor involves shrinking the divide between the business and the IT organization. Reset is an opportunity for both these parties to match business requirements and IT realities to come up with a more viable project plan. The greatest risk to reset success is that the same ingredients that corrupted success in round one will resurrect and spoil round two.

To limit the risk of a second failure, management should conduct a rigorous post mortem or "after action review" (AAR) of the previous project's trajectory. An after-action review (originally developed by the U.S. army) is a formal discussion of an event, focused on performance standards, that enables participants to discover for themselves what happened, why it happened, and how to sustain strengths and improve on weaknesses. Given the degree of difficulty involved in doing a scrupulous self-assessment, it is best to have the AAR facilitated by a trusted, objective advisor. The AAR addresses the following questions:

• What did we set out to do?
• What actually happened?
• Why did it happen?
• What are we going to do the next time?