Everyone claims they want to be agile, but when push comes to shove most people will choose the politically expedient approach over actual effectiveness. For example, what is more important to you: to try to estimate the actual costs of a project and then bring the project in on budget or to spend your IT investment wisely? If your answer is the first then you had might as well stop reading. If your answer is the second then this article is for you, and if you think that it's possible to achieve both then you desperately need to read this article to dispel you naiveté.
Traditional Approach to Budgeting
The traditional approach to Budgeting is to invest the time to develop what is considered to be an "accurate" budget and detailed schedule early in the lifecycle. To do this, a comprehensive definition of the requirements is typically produced during the initiation phase of the project, often simply called the requirements phase. Sometimes a detailed design document is developed, providing more information for the Budgeting process. "Smart" organizations will develop an initial budget and schedule early in the project, then refine it over time as more information becomes available. Not so smart organizations will assume that the initial budget and schedule are official, and actually hold the project team to them. This is particularly true of organizations which take a fixed bid approach to software development, a spectacularly questionable practice at best.
Traditional project teams take a (near) serial approach to development in which requirements are defined and documented early in the project, the requirements may or may not be reviewed and accepted by the project stakeholders, and then they're provided to developers who are expected to build a system based on those requirements. Scope creep, also known as requirements creep, is contained by requiring stakeholders to follow a change management process. Traditional change management processes typically involve one or more people, often referred to as a change control board (CCB), who act as a gateway through which change requests are evaluated and potentially accepted. The goal is to minimize, if not prevent, scope creep so as to stay on budget and schedule. This approach sounds great in theory, but in practice proves to work poorly. In Examining the Big Requirements Up Front (BRUF) Approach I describe Figure 1 in detail, which shows that when you take a traditional approach to requirements elicitation and management that 45% of functionality delivered is never used and a further 19% is rarely used. In other words, a serial approach to development results in nearly 2/3 wastage (and that's only considering the projects which actually deliver into production, so the actual figure is much worse).
Figure 1. The cost of serial approaches to requirements.
Fixed price bids/contracts are bad ideas for both IT and the business stakeholders. Sadly, the business thinks that it's reducing its financial risk with a fixed bid, but the reality is that it forces them into a position where their money is almost always ...