Estimation … Useful Tips

I often receive requests to provide estimates for building huge systems. Typically, such requests only entail the vision of the intended system and the anticipated business value – with limited details on the underlying requirements. The danger in here is that the provided estimates will be used to make important decisions of whether to make the system in house, buy it (if available as COTS), outsource it, or perhaps to cancel the whole initiative in case the estimates are higher that what is feasible.

Definition

To start with, let’s agree that estimation is by definition a judgement, guess or opinion about something. Hence, it it by no mean accurate, yet useful in setting the expectations and helps derive decisions. The estimate can be more accurate primarily if the uncertainties in the business requirements are narrowed down and technical constraints in the environment are identified.

Dealing with Un-Certainties

In Agile projects, no matter how good are the product owner in explaining the product’s vision, or the business analysts in detailing or eliciting the business requirements, there will still be some questions unanswered at the time of estimation. It is only possible to  have all the questions answered and reach eventually to zero uncertainties is at the end of the project.

Nonetheless, the team is required to substitutes such uncertainties with assumptions and estimate against them. An example is whether the intended hotel booking system would give a facility to its customers to pay at the hotel front desk at the time of check-in or not. The team should explicitly assume something, if couldn’t be confirmed at the time of estimation, and estimate on it accordingly.

Estimation Team

It is very important to highlight that it is not the sole responsibility of the Software Architects and Developers to provide the estimates. Product Owner, Test Engineers, Business Analysts are all required to participate in the estimation workshops – and all are members of the estimation team.

Estimation Methods

The estimation team has to start by defining the estimate method they would like to follow based on factors such as, the business complexity, urgency of the estimation request or availability of the product owners and relevant SMEs.

One way to do estimation is by Analogy (Triangulation, Benchmarking) with other developed products, specially if they are related to the same business domain (i.e Aviation, Insurance,…). Estimation by Analogy can also be done with the intended product modules to each other. An example to this is if the intended products can be divided in a number of modules, a reference module should be selected and other modules complexities will be sized relatively to that reference module. The reference module then will be estimated using other techniques, explained below, to come up with an estimate to it and later to be reflected to the other modules based on the identified relative complexity.

Another more accurate method of doing the estimation is WBS (work breakdown structure) through breaking down the product requirements into features that can be estimated individually. The two famous WBS estimation techniques are Affinity Estimation or Planning Poker. I see Affinity Estimation good enough for providing estimate early on for the product development with limited available time for estimation, whereas planning poker is much better while doing release planning where the requirements are more refined and the team structure is defined.

The estimation team should also consider assigning priorities for all the identified requirements, as such priorities will help in deciding on the overall schedule by releasing the product into phases with important features first.

The estimation sheet could look like someting like this:

Requirement Description Priority Assumption Technical Constraints Effort Estimate
 Allow users to provide their feedback  –  High Easy, quick signup We anticipate high volume of requests, but procurement of new servers is not possible… so it is likely to use cloud services  5
Cross check the users legal status with some government entities  –  Low Encrypting data exchange is required Encryption solution should not impact the performance – which is the case now for some existing systems 8

The effort estimate unit can be either a Story Point or Ideal Day.

Team Velocity

The velocity of the estimation team should always be readily available as it should have been calculated based on the number of story points the team is able to burn in an iteration. It will be beneficial if the velocity of the team is categorized based on factors like business domain and adopted technology. The velocity will be used as a reference to when it is needed to convert the effort estimation into Actual Man days (Elapsed Days).

Actual Man Day reflects the reality that typically no one can work on one thing all the time without being interrupted, for instance, to help other colleges or for personal matters like phone calls, etc. So if the ideal day is 8 hours a day, then actual day for any team member could be something like 5 hours.

Estimation as a Social Process

It is essential to know that Estimation is a negotiation process with the business owner(s), and relevant SMEs. It is perfectly fine for them to challenge and question the estimates, and the estimation team has to be ready to provide their justifications based on the assumptions they made and the constraints they could identified. If the business owner is not happy with the overall ‘estimated’ duration, she can simplify some of the requirements, remove some of the identified constraints, work on the assumptions to make a call on them or override them with simpler ones. The estimation team accordingly will revise the estimates and submit them again to the business owner and solicit a feedback to see if they are acceptable or not, and so on.

Estimation is a process that involves high degree of communications, reasoning, planning and courage to speak about the constraints in the organization.

Conclusion

Estimation is needed to help decisions makers in taking informed decisions. Estimation team has to define a clear estimation technique to use based on factors like business complexity and availability of the product owners. The estimates should always be supported by reasons and assumptions. Business Sponsors should accept the fact that estimation is not a commitment from the estimation team, nor it is by any mean accurate.

Leadership in Agile Projects

I’ve been reviewing some academic journal papers and online materials to understand more about the role of ‘leader’ in agile projects (see references). I was astonished how leadership theories were evolved from a basic ‘hero!’ model of leadership to become more of a social process. Looking through the lenses of the social process model of leadership demystifies many of the principles and practices embraced by the agile project management methods currently followed in the software industry.

To start with, the academia revealed two main themes of leadership which are the traditional/mainstream and discursive (socially-constructive) leadership theories. Traditional/Mainstream leadership theories characterize leadership as heroic and ego-centric approach; whereas discursive leadership theories view leadership as a social process between people which emerges through interactions and daily discourse. Mainstream leadership theories capture a simplified and coherent version of the world and position the leader as a discrete individual who can change situations by applying leadership techniques, principles or strategies. Leaders can be in a complex, un-certain and un-predictable situations where such simplistic view of leadership does not necessarily help in overcoming such situations (Cunliffe and Erikson (2011), Collinson (2005) and Fairhurst (2009)).

Such a stance of embracing ‘leadership as a social process’ is recognized and supported in the software industry by the agile movement. The Agile Manifesto, written by some of the world’s prominent software practitioners, states explicitly that building and delivering software applications is better to be done through self-organizing team; the team that reflects on how to become more effective, then tunes and adjusts its behavior accordingly (agilemanifesto.org, 2001). Scrum agile framework described self-organizing teams as teams which “choose how best to accomplish their work, rather than being directed by others outside the team” (Scrum Alliance, 2014). Takeuchi and Nonaka (1986) work that had been published in Harvard Business Review and inspired the creation of Scrum agile framework (Sutherland, J., 2011) stated that a team possesses a self-organizing capability when it exhibits autonomy, self-transcendence, and cross-fertilization.

Although the agile community advocates that agile team is a self-organizing team that exhibits shared/distributed leadership amongst its members, which is a shift from the traditional model of leadership; but still the industry is not clear on how to empower the self-organizing team to exercise such a shared/distributed leadership and under what circumstances to shift to the traditional leadership approach. Neither the Agile Manifesto nor the Scrum agile framework specify what role is responsible for, and how to, building trust and harmony between agile team members to enable them as a team to direct and control the project autonomously and collectively. Many industry practitioners tried to fill this gap by suggesting a leader from outside the team who gradually gives more control to the team once they establish trust between them and prove they can communicate effectively (Mandarino, P., 2012). Some other practitioners advised on appointing an agile leader only in case of projects that are complex and big with many teams involved, or in cases of cross functional projects that may have an impact on the culture inside the organization (Kniberg, H., 2015). The agile leader guides the evolution of behaviors that emerge from the interaction of the agile team members but not specifying in advance the behavior of the team (Cohn, M., 2010); and the agile leader needs to work across the organization to create a work system that enables teams to deliver value to customers and the organization (Esther, d., 2011). Some practitioners believe that this agile leader is actually the Scrum Master role itself (Cohn, M., 2010) which only works as a guide to the team and is not supposed to influence how the team will go about building the intended product (Scrum Alliance, 2014). Others see the need of different types of agile leaders based on the context, such as agile project manager (Mandarino, P., 2012), agile manager, agile program manager, agile account manager or just an agile coach which is typically the Scrum Master (Johanna, 2014).

My view on this is management needs to empower development teams to be true agile, self-organizing teams who can autonomously and collectively direct and control the product development. Until that fully happens, there should be a transitioning period with the help of external agile leaders, such as agile project manager, who should help the agile team in creating an environment of trust and high collaboration without influence team’s decisions on how to build the product. Management role in general is absolutely needed even for fully self-organizing agile team as management is responsible for defining constraints around the product being built, marketing the product once finished and motivate each individual while the project is undergoing.

REFERENCES

Agile Manifesto (2001), ‘Manifesto for Agile Software Development’, available at: http://agilemanifesto.org, viewed on 06 August 2016

Cunliffe, A. and Erikson, M. (2011). Relational leadership. Human Relations, 64(11): 1425-1449. DOI 10.1177/0018726711418388

Collinson, D. (2005). Dialectics of leadership. Human Relations, 58(11): 1419-1442. DOI 10.1177/0018726705060902

Cohn, M. (2010), ‘The Role of Leaders on a Self-Organizing Team’, available at: https://www.mountaingoatsoftware.com/blog/the-role-of-leaders-on-a-self-organizing-team, viewed on 06 August 2016

Derby, E. (2011), ‘Misconceptions about Self-Organizing Teams’, available at: http://www.estherderby.com/2011/07/misconceptions-about-self-organizing-teams-2.html, viewed on 09 August 2016

Fairhurst, G. (2009). Considering context in discursive leadership research. Human Relations, 62(11): 1607-1633. DOI 10.1177/0018726709346379

Johanna (2014), ‘Which “Scrum Master” Are You Hiring?’, available at:http://www.jrothman.com/articles/2014/09/which-scrum-master-are-you-hiring, viewed on 12 August 2016

Mandarino, P. (2012), ’Leadership in An Agile Environment’, available at:  https://www.thoughtworks.com/insights/blog/leadership-agile-environment, viewed on 07 August 2016

Scrum Alliance (2014), ‘The Scrum Guide’, available at: https://www.scrumalliance.org/why-scrum/scrum-guide, viewed on 06 August 2016

Takeuchi, H. and Nonaka, I. (1986), ‘The New New Product Development Game’, available at: https://hbr.org/1986/01/the-new-new-product-development-game, viewed on 06 August 2016