30 Dec 2012

Agile's Prerequisites, Risks, Limitations and Weaknesses

Agile's Prerequisites, Risks, Limitations and Weaknesses


Often the question is asked when to use Agile and one or more Agile methodologies and methods.

To make this decision, it doesn't suffice to know the strengths of the Agile approach (philosophy, attitude, methodology, work environment, ..). It is essential to understand its prerequisites, its risks, its limitations and its weaknesses.

Let's get realistic. If the Agile approach is not a silver bullet, then they do exist. One size doesn't fit all.

However, these prerequisites limits, risks and weaknesses may depend of the chosen Agile methodology, the specific Agile implementation in organisation and of course the people themselves. Possible, local solutions can be found to circumvent the disadvantages.

This list is based on my experience, understanding and readings. It is perfectly possible that other people come to a different conclusion. This list is not exhaustive and is provided as is. It can serve as a starting point for further investigation.


1) Preconditions for an Agile approach
 
Budget
 
  • ...
Client
 
  • Availability: Daily involvement and face-to-face meeting (possibly through videoconference)
  • Accepting not to know what will be delivered for a certain budget (although, most valuable and risky features are delivered first, but content of later sprints can change)
  • Accepting a different way of following up the project progress
  • Should provide a product owner (on business oriented software projects)
  • ...
Team
  • Team members be willing/motivated to adopt the Agile approach
  • Must be highly skilled
  • Must be disciplined and possessing the ability of self-management
  • Must have a team spirit
  • Must be able to work with the customer, end-user and other non-It people (interviews, workshops, ...)
  • Project team must be Multi-disciplinary. ==> Programmers must possibly be trained if the "developer" combines the roles of analyst, software engineer, programmer and tester.
  • Must master the Agile philosophy, the chosen Agile methodology and tools.
  • ...
Organisation and Work environment
  • Presence of a product owner who should be well-trained in informatics
  • Presence of a Agile coach to facilitate collaboration and enforce the Agile philosophy and methodology
  • Project Manager should accept his/her role has changed
  • Analysts, if present, should accept his/her role has changed
  • Place for daily stand-up meeting, possibly also for the daily meetings with business stakeholders
  • ...
Project Type
  • Smaller projects
  • Frequently delivery must be possible (can be argued)
  • Existing systems are more suitable than new systems.
  • Local projects. Projects with ramifications accross the organisation should be avoided (uncontrolled impact elsewhere in the company).
  • Projects with many different business stakeholders may be more difficult in Agile.
  • Projects including different systems may lead to more than one product owner.
  • Should have a stable, mature overall architecture and core of the system. The uncertainty about content and changes should (mainly) reside in the peripheral logic (detailed business logic, end-user features, outputs like screen and reports). For example: Through creativity end-users features can be adapted or created.
  • ...
 
Product
  • Must be deliverable in parts (eg separated features)
  • Large business critical systems should be avoided
  • ...
 
2) Limits of the Agile Approach
  • Size of the team between approximately 5 to 15 persons
  • Agile is difficult scalable. Most methods are oriented to small teams working fast. Larger projects will require an overall architecture and more planning. This may become lesser adaptable. This discussion is going on in the Agile world.
3) Risks of the Agile Approach
  • An Agile approach is hard to implement
  • It is very demanding of the team members (to reach quality and excellence)
  • Since reuse is difficult, a same code may be developed accross the company. This reproduction of a same logic constitutes a risk of inconsistencies.
  • Depending on the product owner, there might be a risk for suboptimisation.
  • There is little control on the impact outside the scope of the system, elsewhere in the company.
  • Team members leaving the company/organisation is a risk if too little documentation.
  • Since each change is a cost, repeated or excessive changes may kill the recovery of the investment.
  • If the Product Owner is a business person without sufficient competencies in informatics, he won't be able to make sure the software system produces business value. His decisions has a tremendous influences on the system and on the project and may then put the project in danger.
  • ...
 
4) Weaknesses of the Agile Approach
  • The Agile values can easily be abused for limiting or avoiding documenting, planning, respecting the contracts or for doing some administrative tasks.
  • Since sprints are short and a frequent delivery is promissed to the customer, the danger exists that no time is left for decent analysis, thorough testing, appropriate documentation (which is lesser valued), refactoring (what the client doesn't see and from which he/she doesn't get benefit) and learning.
  • If client the continuously changes his mind, there is a lot of rework and refactoring. Refactoring is a kind of rework that doesn't produce immediate/direct business value (client has to pay for it) but which is needed to keep the code well-organised, clean and clear.
  • Testing thoroughly is difficult under an Agile approach.
  • In larger companies, releases may be organised on a quarterly basis. This decreases the value of delivering early. Delivering frequently, however remains usefull to get faster feedback by using it as delivery of functioning software for a demo or user test.
  • It is tempting to simply develop whatever the client asks for. This approach limits the usage of the full potential of IT.
  • As Agile approaches are based on the business demand and on the instructions of the product owner, the approach is reactive in nature and increases the dependency of IT from the business community.
  • Depending on who does the analysis/design, there might be a lack of innovation at conceptual level.
  • It might be difficult to have enterprise-wide models if the product owner is responsible only for what happens within one/his department/organisational unit.
  • The focus is on the business logic and on the feature of the software since this is what creates (immediate) business value. But how well does the Agile approach supports the needs and requirements of the IT departement. And how well attention got the non-functional requirements, the "ability"-qualities (= ISO9126, ISO/25010).
  • Risk management is limited to the sprint, unless a broader and overall picture exists. 
  • ...

27 Oct 2012

When is a Project Successful ?


A seemingly simple but essential question is when is a project successful? The usual and probably most evident answer is when it delivers on time, on budget and within scope. But is this the right answer?
If the project reaches its objective, it is successful. If not it is a failure. Objectives allow people to work together in a same direction on a same mission. But more than that, people are evaluated on their ability to reach the established objectives.  Rewards are often linked to objectives. Erroneous objectives and a improper reward system based on these objectives may cause a lot of havoc in the company. We shouldn't take it lightly.
First, let's take a closer look at the iron triangle objective.


The Iron Triangle

A project which delivers late or exceeds the budget is not successful. The established time and budget are compared with the project execution. If the comparison is false, then the project is a failure.  

established time and budget   <,=,>?   project execution

If the project isn’t successful, we assume the execution wasn’t right. If the comparison turn out to be wrong, at least one of the two sides of the comparison must be wrong. So what if the project has been executed efficiently and effectively? What if the right side of the comparison is right and the left side is incorrect? After all, time and budget are only estimates. What is more likely to be wrong, the estimates or the project execution, which is the reality? Can we assert the estimations to be absolutely right and the reality to be wrong? Are estimations more important than the obtained results?
 
Time and budget are estimated too low.

The persons doing the estimations are too optimistic. Maybe they wanted to save money. However, thee estimations of time and budget are too low.

The team members may seek to respond to the expectations. They want to preserve their reputation. They want to be evaluated well and get rewards and promotions. Whatever the reason, most team members will do their utmost to achieve the objectives. They will do what is needed to deliver on time and maybe also on budget while respecting the scope, even if the estimations are too low. Undoubtedly, this will put the project under pressure. The lower the estimates, the greater the pressure, unless the estimations are more than ridiculous low.

To be successful, the team has not much choices. It must apply tactics like trusting the demand and information of the business community, working overtime and taking short cuts. Maybe, the product will respect the scope, but it isn’t sure it will be of great value to the business. The demand, the scope, the requirements and the specifications will be interpreted in such a way that the minimum will be implemented to comply with them. Anything that is not specified, even if needed, won't be implemented. The most simple version of the solution will be built. The quality of the product will suffer. The result is a too simplistic, poorly designed solution, having awkward or lacking features. Important characteristics, like configurability, maintainability, flexibility or evolvability will be weak. This will increase the operational cost, as well as the cost of future changes.
The work climate is likely to deteriorate and conflicts may arise between the team members.

Using time, budget and scope increases the risk of problems, increase of cost and/or prepare for a more difficult evolution.

The project can be delivered on time, on budget and accordingly to the scope but :

·         the project members may be frustrated or completely exhausted at the end of the project.
·         the delivered quality is poor.
·         no business value has been created.
·         although the specifications has been perfectly respected, many issues are created possibly creating downtime. The data is unmanageable or unreliable. Clients are lost. It loses credibility. Reputation of the company decreases.
 
Time and budget are estimated too high.

A project manager has to be responsible. He doesn't want his project to fail, or to be perceived as a failure. He/she wants his/hers project to be successful. That’s very easy. He/she simply provides estimations that are x times more than needed. The project delivers successfully but much time, budget, resources and opportunities have been wasted.
 
Measure of the Iron Triangle?

What does the iron triangle measure? In fact, the iron triangle is not a measure of success of the project at all. It is at its best an indication(!) of the performance of the project manager to estimate the project and manage the execution against the agreed estimations. Why is it only an indication? It doesn't represent with 100% certainty his/hers performance. A project manager never has a full control over what happens in the project.

Delivering within the iron triangle

Delivering within time, budget and accordingly to the scope makes more sense when it is feasible. Therefore, certain conditions must be satisfied. The product to be built must be fairly well known. All the specifications must be detailed enough. The estimations must be reliable and realistic. Moreover, no important incidents may occur. But even then, other criteria can be added.

Not all projects are the same. Installing a technological infrastructure or an in-house software development project are completely different projects.

During most software development projects the client's needs, the problem and the context are investigated. This investigation may be done partly before the project begins and continues during the project execution. For these projects, the product is not perfectly known when the project starts. Estimations are then more unsure.

Time, budget and scope can be variable. They can be regularly reviewed during the project execution. The sponsor and client may not like this. He/she is more likely wanting to know what a project will cost, what is the delivery date and what he/she will get before approving the project start. However, in some sense, Agile projects works with variable constraints. It is presented differently and not named like that. The time, budget and scope are established per iteration and is established just before the start of the iteration. The total scope, duration and budget is the sum of the scopes, durations and budgets of all the individual iterations. Thus, this total scope, time and budget of the whole project varies on every iteration. This makes it easier to deliver within these criteria.

Project Success Criteria

Do we really need to know if a project is successful? Although this may be interesting, it is maybe not what we really seek to know or what matters most. This will be clarified.

The PMI defines the project as a process. A project can be defined as successful when it efficiently produced the product it is intended to produce.

Using only success criteria based on the process (project execution), doesn’t put much responsibilities about the final product on the shoulders of the project team. The project team can be judged successful because it produced efficiently the product demanded by the sponsor and/or by the business community. Nevertheless, the product can be completely unusable for the business or may even be harmful for the business. Basically, some project management skills and technological knowledge suffice to achieve this. This approach has clearly detrimental effects.

Product Success Criteria

The sponsor wants to get, as return on his invested budget, a product that suits him. His investment is motivated by the result, not by the building process (project). The product is more important. It makes sense to deduce that the success criteria should be based on the project’s product. The question has then to be reformulated as “Is the produced product successful?” To be successful, the team need to understand the operational environment and how people will use the product. Technological knowledge doesn't suffice anymore. More skills, like analysis and design skills, are needed.

Hereby, we must note that, measuring its success at the end of the project is just a snapshot of its business value. During the lifecycle of the IT system, after the project ended, its business value will continue evolves.

Business Success Criteria

The product can be useable by the end-users, it has to contribute to the business as well. This matters more. It's not a bad idea to link the project to business objectives. The project team will then be contributing to these objectives.

But again, things are not that simple. There is rarely a one-to-one relation between the business objective and the project’s product. The product created by the project contributes to reaching the business objective. But it is not the only factor. For example, a new sales support software application contributes to increase the sales. But the marketing campaign and the sales force influence the sales volume as well (and probably even more than the software application). Factors, external to the project, do play an even greater role. Reaching a forecasted higher sales volume is not a direct measure of the project’s product success.

 

These criteria require the IT team (not all the team members) to have some insight and knowledge in the business. It also requires a greater information exchange and some more collaboration between the business people and the project team.

One more level higher

Beyond the level of the business, there is one higher level: the enterprise. As the IT department should contribute to the enterprise, it is (also) from that level that objectives and criteria should be derived. This doesn’t only require additional knowledge and skills, it also re-positions the IT department.

Selecting Success Criteria

Success criteria should be selected with care. The most obvious criteria are derived from the objectives (project objectives, product objectives and business objectives). The facilities and machines have an accounting value. Similarly, the IT infrastructure and the information systems have a business value. Augmenting this business value can be used as a success criteria as well. We may also look at the more global desired outcome like end-user satisfaction and client satisfaction.

Or, we may get inspired from the resource-side of the initiative. Through projects, new skills can be acquired. There are factors that may seem to be not important to measure the success but which may undermine future successes. The resources may not be exhausted at the end of the project. They should be fresh to start on a new project. Unorganised or unreliable documentation or an inelegant design may not be a barrier to achieve the present objective. But they may be an obstacle to reach future objectives. Today, the failure or the success of tomorrow is prepared.

An organisation can establish a catalogue of success criteria by analysing projects and brainstorming. For every project, it can then consult this catalogue to establish a list of criteria for that project. Then, it can add more criteria specific to that project. The organisation can do the same for the product.
This is quite a lot of effort. But, why do we want to know whether a project was successful or not? It is important to know it. But today, in practice, except declaring a project’s success, what is really done with knowing that a project was a failure or a success? I have heard about failed projects that where nevertheless declared as a success and even project managers of projects that were admitted to have failed to be promoted.

17 Oct 2012

CIT’s Problem 7.2: The Present Evolution of the IT Department


IT is a domain in full evolution. Since a few years, we can observe some on-going tendencies.
·         Outsourcing
·         Cloud computing
·         BYOC (Bring Your Own Computer), BYOD (Bring Your Own Device), BYON (Bring Your Own Network)
·         Shadow IT - End-User Computing (EUC)
·         Agile

Using new products, concepts and services available on the market is fine. However, it is crucial that the company benefits from it. Some companies may, as a need appear, as a difficulty has to be overcome or as a problem must be solved, systematically look to buy or acquire a solution. We may acquire/buy training sessions, consultancy, services, frameworks, standards, frameworks, models, software, tools, technologies, … even without first really investigating and understanding the problem and without trying to solve it ourselves.

Cost saving, dealing with peaks in work load, lacking of some specific competencies, the transfer of risks or a faster availability of the solution are certainly valuable arguments. But the decision to buy may also be a default way of operating or the decision may be driven by some underlying motives. They might be symptoms of underlying problems. Therefore, it is crucial to bring them to the surface. Some possible underlying reasons are:

·         The business community may have lost trust in the IT department.

·         They find that the IT department is delivering to slowly or they impose to rigid conditions and restrictions on security and usage.

·         The business community feels confident enough to take care of their own information needs. Why shouldn’t they take an initiative. They didn’t think about it to discuss the issue with the IT department.

·         They find it important to be more independent of the IT department. A power struggle may be going-on.

·         The IT department wants to avoid having to invent some concepts locally. They may have a belief of not having the skills or have more trust in concepts invented elsewhere. (driven by fear, avoiding risk taking, lacking of skills in problem solving, lack of trust in own people).

Of course, these are not the only possible motives.

The tendencies described above are not without drawbacks for IT department and indirectly for the company.

·         By looking elsewhere for solutions,

    • the IT department may lose valuable competencies (competencies in business informatics, systems analysis, ..)
    • and as the knowledge of the implemented solution decreases and comes into hands of a third party, they may lose control over the implemented IT solutions and about the final cost.

·         Solutions implemented outside the umbrella of the IT department are unknown to this department. They are not supported. The data may and business continuity may be at risk. It may create breaches in the security. And so on. Globally spoken, they lead to unmanageable IT implementation (chaos).

·         A shift of responsibilities from the IT department to the business community occur. It doesn’t need to happen in a formal way. If it happens because an actor starts to take some decisions and makes some demands, the shift occurs implicitly. But when the company has to face IT failures or poor results and high costs, the IT department may still be blamed.

A shift in the role of the IT department will occur. They will have much more to manage external suppliers of IT products and services. They will be held responsible of maintaining solutions chosen, acquired or developed and implemented by others. And in the end, one way or another, they may be held responsible for decisions taken by the business community concerning as well.

All this continuous to undermine the position of the IT department. And, instead of improving the capabilities of the department, it will weaken them.

> Next post will probably concern the role of the IT department that allows to fully exploit its capabilities.
 

CIT’s Problem 7.1: Present Role of the IT Department


The role of the IT department is so critical for a company that it has to make sure it takes the greatest advantage of it. When we look at the present role of the IT department, this is not what is happening at all. And it’s not simply a matter of working harder or smarter. It’s a matter of role definition, position, responsibilities and developing the right competencies. We need to take another look at the IT department.

This difference will become clear when comparing the present role of the IT department with a new role. However, today, the IT department is evolving towards a third role. This has also to be considered.

Today’s common role of the IT department


The following responsibilities and activities are commonly associated with larger IT departments. Smaller departments may have a subset of these responsibilities and tasks.

The role of the IT department is to provide technological solution to the whole company regarding electronic information. The IT department is in charge of all computer based systems.

The IT department develops and implements these IT systems. It maintains, manages and optimises them.

The IT department provides custom-programmed solutions. Company’s websites can be included here.

The IT department provides, maintains and manages the office automation software (e-mail, word processor, spreadsheet, sharepoint, fileservers, ..).

The IT department defines, implements and manages the needed technological platforms and work environments (development environment, test environment, ..). It should provide the best and quickest operational environment for their end-users.

It also provides the infrastructure for automation, which is the computers, the network, the circuitry and all equipment needed to make the IT systems work.

The IT department is also in charge of securing all these systems and networks.

It is responsible for data quality assurance.

It provides assistance to the end-users regarding the software, systems, tools and devices it (the IT department) is responsible for.

The IT department is also responsible to solve the technological issues the company may face. They have to come up with solutions to for users.

The IT department is responsible to ensure the continuity of the implemented IT solutions and infrastructure. In case of system failure, its role is to restore and to fix the issue limiting the impact on the business and its activities.

The IT department may provide policies, frameworks, guidelines and parameters for the individuals' and operating units use of software systems, computer systems and networks, including the encoding of data.

The IT department acquires software applications, technologies and devices which may enhance the functioning of the organization.

Additionally, the IT department also:

·         manages the data centre operations

·         manages programmes, project portfolio and projects

·         operates the Project Management Office or Project Office

·         manages the access to the IT solutions

·         does facility management (equipping and managing the computer room)

·         implements, maintains and manages solutions for telecommuting (teleworking)

·         follows trends in IT

·         does research in IT solutions

·         implements, maintains and manages telephony, fax machines, printers, plotters, video conference, ...


Descriptions of the role and activities of an IT department found on the worldwide web sum up these activities. Almost certainly, they represent the expected role of the IT department. And probably, they also represents the largest part of the IT department’s activities.

If this is the case, it limits the role and contribution of the IT department. Plenty of competencies that have to be found in an IT department are not utilised.

Before building a solution, the problem, its context and the objective need to be understood. The solution must be designed. It may implement innovative concepts. But in essence, the listed activities are all oriented to building, maintaining and managing solutions. They suppose that, to some degree, that the right solution has already been defined.

By limiting the role of the IT department, and not utilising all competences to be expected in it, the risk to have solution of lower quality or, worse, one that simply doesn’t fit increases. Achieving business-IT alignment and innovation remain a very far dream.

Some activities that have been forgotten to be listed: organising the data, data migration, ensuring the business continuity and disaster recovery at the level of the IT implementation, the development of frameworks and methodologies.

But more important, this description of the IT department overlook domains like enterprise architecture, business analysis, business process modelling or information architecture. These domains are still rather young. They don’t provide a direct benefit to the business. Nevertheless, they are key activities for an IT department.

13 Oct 2012

CIT’s Problem 6: The Position of the IT Department

The role of the IT department may be to support other business functions. It’s a second class role.
The IT department is only considered as a cost centre.
The CIO is not always considered as one of the main managers of the company. For example, he/she might not be invited when the company’s strategy is established.
The IT department copes with a wrong system belief about IT, about the IT people and about the IT department.
Most non-IT people do not understand the IT department and IT.
There is a communication gap between the IT department and other business department.
Furthermore, the IT department has to face the pressure coming from the business, limited resources, technology that gets older in only a few years, continuous innovation of IT, technologies and products on the market and the existence of legacy systems.
The IT department must organise itself, for example define, learn and improve its processes, methodologies and methods.
It is hard to find really skilled people in IT.
The IT department must train its people. Its people should improve their skills.
The IT department has a bad reputation.
The IT department bears an enormous responsibility. Information systems cost a lot in building and in keeping them operational. A well designed information system can maximising its business value. On the other hand, a badly designed system can increase the cost of operation as well as the cost to allow the system to evolve. A failure of a business critical application or computer may cost the company really huge amounts.

CIT’s Problem 5.2: Obstacles for the IT Department in the Common Approach


The approach discussed in CIT’s Problem 5.1 puts the IT department into a specific position and situation. These are discussed here.

The IT department has no control over the process.
From a certain point in time, information is available (eg as plan, objective, mismatch between objective and operational results, complains, operational data, ..) or symptoms or consequences (tendencies, behaviour, problems) can be noticed. The issue can be detected. From this point in time until the submission of the demand, the business community can act. The detect the issue early and act promptly or they can stretch this period. The IT department is completely absent in this phase.

Random chronology
What determines the order or timing of the demands? Unknown issues can’t be solved. The issue must be detected. It is, in essence, this detection  that is the trigger. But the detection happens by coincidence. The timing and the order of detection of need is completely anarchic. This order and timing can be adjusted by the moment some critical decisions are taken and by the priorities the business assigns to the issues. The metaphor of constructing a house can be used here. This future house owner asks to build first the roof because it is raining. Then when he needs to cook, he feels hungry and needs to cook. He needs a kitchen. So he asks to build a kitchen. And so on. Houses aren’t build like that, neither are information solutions.

Piecemeal built enterprise IT solution
The demands can be triggered by information needs, by operational issues or by ideas of improvements. These demands are not planned. By responding to these demands, in fact the enterprise IT solution is built by solving one need or problem after another. It is conceived and build piecemeal. It leads to a patchwork of individual and local solutions (may create the need for EAI-solutions). Once built, the local solution is integrated into the whole. Or, if it isn’t possible, it is connected to other existing parts. It is a post-design integration, instead of an integration by design. It is like constructing a building room after room without having any idea of how many rooms and floors will be needed, of how the building has to look like when it will be finished or even of the intended size and purpose of the building. After a while, when many room are built, one can infer what the final solution will look like. The solution grew from the start by adding components and continuous transformation. It has never been engineered as a whole system. Then, it might be a good idea to engineer it to increase its qualities.

The IT department runs after the facts
The whole approach reactive since its philosophy based on a response to a demand.

The existence of an issue or need emanating from the operational level is (mostly) required to solve it. The issue must exist to be detected and before taking actions. This is late. The issue exists already and hinders the business. They can’t or are seldom anticipated. This creates a pressure and makes the solution systematically to come late.

The situation is different for needs generated by a changing environment of the business (outside the company) or by the strategy and other plans of the business. The business community may be direct its attention to what they are in charge of, to those aspects they know and to the activities they will have to perform to execute the strategy. Information needs are not really of their responsibility. They may not pay attention at the future information needs. They have no clue of the impact and time needed to solve them. Moreover, the change must first be clarified and solved at the business level before being able to define the exact information need. (This idea is not completely true.)

A demand is submitted for a need that will materialises within 5 months. But developing a solution in decent conditions takes 9 months. The demand has been submitted too late. The demand should be submitted early enough to give to the IT project that amount of time estimated by IT experts needed to execute the project.

In both cases, the IT department responds to the demands from the business community. The degree of reaction versus planned work depends from organisation to organisation.

The IT department is fully dependent of the business community
Projects are based on the business demands. The project team tries to deliver what has been asked by the business. This approach creates an important dependency in time and content. It makes the IT department dependent from the business community for an important part of its activities. The business value, the quality and the impact on the business of the solution produced by the It department are directly dependent of the quality of the demand and other information delivered by the business community.

Also beyond the demand, the business community provides the main input to the project team.

The business community may filter information and demand.

They may want to start with a simple and small solution or they may consider to consider only the main case. Once this is build, they may decide to let the solution grow by adding new requirements or features to the original demand.

What can be filtered:
        ·         Features and requirements of lower priority
        ·         Aspects that are not clear yet to them
        ·         Parts that are more complex
        ·         Requirements that appears only as an extension of already formulated requirements
        ·         Lesser frequent cases and exceptions
        ·         Alternative paths

A few examples:

  • We may start with requiring a raw text report. Later, requirements can be added to have pictures, configurable data on the report, configurable page layout and paging features.
  • The requirements may state to have a single-user system. Once this proved to work, the client may ask to make it multi-user or an access from anywhere through the web can be requested.

The business subject matter expert may belief some information is not important and keep these details for later. This filtered information may in the end be very important for the design of the solution or for the choice of technologies.

The danger of filtering is that it is done without consideration for the consequences on the way information systems are or should be built, on the impact on the conception of the solution.

The IT department is last link of the chain
The whole conception of an information solution is a process. But, and particularly with the present approach, it can be considered as a chain. The first link consists of the issue itself. This can be the publication of new legal requirement, symptoms and consequences of an operational issue, the idea of an improvement, and so on. This information must be noticed and considered by the business community. The business community is the next link. Then information is transferred to the IT department. They are the last link. In this arrangement, the It department, as being the last link, always lags behind. This is not favourable to “solve” the business-IT alignment challenge.

The IT department is the slowest chain
The business domain is (usually) a world at the level of the human. On the other hand, the IT people often work at a very detailed and abstract level. Since computers can’t interpret and decide by themselves, a huge part of the work is very formalised. It should be complete and logically consistent. A change at the business level can be expressed by changing a single word, adapting the formulation of a phrase or paragraph or adding one. If an IT solution must be adapted to such a “minor” change, this may mean work from a few minutes to ultimately months or even years.

Conceiving information systems is a labour intensive work. But it can’t be done well when changes arrive at pace they can’t keep up with.

Many smaller changes may keep the business healthy. As they imply a lot of work to adapt the IT systems and as these adaptation are preformed under pressure, the changes are implemented with as single goal, to have things functioning as expected. The many changes implemented with this objective as norm, weakens the system. Over time, it makes a mess of the IT implementation.

There are techniques to allow a greater flexibility. Some (not any) changes can then be implemented more swiftly. Nevertheless, the IT department is a very slow component in the company.

This means that it may be easier, or at least it requires lesser resource to implement a change on the business side than to implement the change in IT. Too many changes imposed on IT will increase the pressure in this department.

The present approach creates a lot of obstacles for the IT department to deliver.

10 Oct 2012

CIT’s Problem 5.1: The Role of the Business Community in an Inefficient Approach

Today, the business community has quite a lot of authority and influence on the implementation of information solutions. Although, this influence is mainly exercised in the context of projects, it goes well beyond as well. The involvement of the business community is essential. The questions are what effects does this influence has on the final IT implementation and if the present way of working is beneficial.
Is there a relation between the influence of the business on IT projects, the present way of working and the fact that IT departments struggle to deliver and to satisfy the business ?
Let’s take a closer look at the course of events happening from the very origin of information needs to their resolution and even beyond. The picture that is drawn describe a typical evolution of the situation.
The present common approach of software development and the inefficiencies:
(click on the picture to enlarge)
 
1.      Detection of information needs
Quite often information needs reveal themselves through symptoms and consequences. Presently, the business community is the first party to notice or to experience them.
Symptoms may first not be recognised. Some symptoms require an experienced eye to be noticed. On the other hand, consequences having an adverse effect on the business, are much easier to be noticed. Unfortunately, those consequences may be interpret as a problem. Then, the real problem may continue to remain hidden. Business people facing the consequences may individually try to solve them. Each one may look for his own solution. Then people start to complain. The issues are discussed among colleagues. Some colleagues may start to adopt a common solution. Since the problem is identified as more important, a business person with some technological skills may find it worth to try to build an ad hoc solution. Since it is not his main competence, the solution may have serious weaknesses. Sometimes it may work. Sometimes it may work only for a while. It may also create new issues. The more the business community uses such non-solutions, the more time it will take before the real problem is addressed with a well-thought solution.
As the problem is apparently solved, it will take some more time before the business community becomes conscious of the existence of the real need.
Recognising the presence of an issue is not the same as identifying the issue or as understanding the issue. One can know that something is wrong without being able to pinpoint the real problem.
2.      Officially admitting the existence of information needs
As difficulties persist, the issue can be formally recognised and put as an item on the agenda. The issue is officially discussed. Meanwhile more time passes and no decision is taken yet.
The problem continues to impact the business and, over time, even to grow. Non-solutions are still in use and may even proliferate. The business community feels responsible about the issue. They want to do something about it. But they also feel not confident in how to solve the issue, in how to give it a decent solution. This is normal since conceiving information solutions is not their area of expertise. Finally, because of the urgency, a decision is taken. The business community must formulate a demand to the IT department.
Other information needs may be generated by plans. But since, the business community may focus their attention and effort to other aspects more related to business activities, information needs may be discovered only late in the execution of the plans.
3.      Elaboration of the business demand
A business subject matter expert formulates the demand. Different actors from the business community must agree with its content. More time passes. This demand should express a need. Often, it doesn’t. Instead, it describes the solution that is assumed to solve the need. To do so, the business community has conceived the future solution. This can be through the construction of a mental picture of the solution.
Although it is undisputable that the subject matter expert knows the business, he/she lacks some essential competencies. The business community has developed competencies needed to perform business activities. Competencies in analysis and in designing information solutions are not needed to perform business activities. They haven’t been developed. He/she may work with a wrong mental picture of IT in mind. (see post “CIT’s Problem 2: Business Mental Picture and Reference Framework”). The business experts may be under pressure to formulate the demand. He/she doesn’t have time to reflect more deeply on the problem and its solution. Moreover, the business subject matter doesn’t know what the IT department and the project team need to know to (possibly) conceive and build the solution.
Because of the lack of decent analysis, there are still plenty of unknowns. But, the business community cannot deliver a demand that is incomplete, can they? The business subject matter expert makes choices. These choices aren’t based on expertise in information system design. They might be based on what he/she feels is the best solution or on simple preferences. If during the project, or even later, it turns out to be wrong, change requests can be formulated. The IT guys will have to adapt their software. Software can always be adapted. The commitment of the business community on their demand will be only very temporarily. The business demand is thus questionable. (Cfr: post “Do You Rely on the Demand and Requirements from the Business Community?”)
4.      Submitting the business demand to the IT department
This demand is submitted to the IT department. So far, the business community did not think too much about the time needed by the IT department to build a solution. This point pops up now. The time and resources needed to build the requested solution is one of the first concerns of the IT department.
5.      Planning the project
The IT department has already different projects on-going. Its people works already under pressure. The It department has to see how it fits into their planning. Maybe within M months they have enough free resources to start the project, at least, if other projects do deliver on time. Or, they may hire external people and then they can start earlier.
The IT people estimate the size of the demanded project, based on the business demand. They estimate it to X man/months, Y months of project lead time, and a cost of Z. Because of the distorted image non-IT people (and even IT people) do have of IT (see post “CIT’s Problem 2: Business Mental Picture and Reference Framework”), they can’t understand why it would take so long. If they would have to do it with their tools (eg with programmable environments and macro’s), they would do it much faster.
Much time have already passed (the word “wasted” won’t be used yet). Meanwhile they suffered from the information need. Now, their patience is up. The business community needs it urgently. The business community is the sponsor of the project. They are the client. It must be delivered as soon as possible. The pressure is maximal.
The time and budget are cut by half. They are agreed to be X/2, Y/2 and Z/2. But IT people may be rather optimistic and confident in their skills. The actual estimation may well have been 2xX, 2xY and 2xZ.
Since the project is urgent, the decision is taken to start the project and to execute it in parallel with other projects. IT people will have to work on different projects simultaneously. Multitasking is certainly not an efficient way of working.
6.      Relying on the business demand
The analysts, architects and software designers work under pressure. The business community formulated a demand. They expect the IT department to build what’s in the demand and nothing else. Thus, they will get what they asked for. At least, the IT project will try to build and deliver this.
Putting the demand of the business community in question means more work and additional time. The project team has no time whatsoever and nobody expects it to do so. Consequently, if the demand is based on a wrong insight of the real need and of the situation or not very well-thought, the delivered product won’t solve the information need and/or may create new problems. Delivering the described solution is the objective imposed by the business community.
7.      Project progresses. Maturing Ideas. Changes ahead.
As the project progresses, the project team must go into greater detail. Unclear aspects must be clarified. More decisions, postponed so far, must be taken and additional answers must be given. At the same time, the insight in the situation and in the needs increases. Ideas become more mature. New ideas come up. As a result, the scope and requirements may change. They change not because the environment or the objective changed, but only because the situation is more clear and the understanding improved. New features are demanded. These changes make the project’s estimations of resources, time and cost worthless. They must be reviewed. It also means a lot of rework for the project team.
A request that, for the business appears as being simple, eg allowing multiple users to work simultaneously or changing the data structure, may have a huge impact on the system. The business community can’t evaluate the impact on projects, neither in terms of changes and side effects, nor in terms of work volume.
Design decisions made on the early requirements may become invalid. And it may even happen that the architecture or the chosen technologies become unsuitable. Changing one of them means throwing nearly all the work that has been done so far away and restart from scratch. Ultimately, delays and cost may skyrocketing.
If the business community keeps on changing its mind, the project may become a never ending story. It provides work to the IT department but without delivering a result.
The business community may adopt the philosophy of trial and error, of doing what they like or what they think is needed, of following their preference without a lot of thinking beforehand, of experimenting, of exploring and of discovering. They can decide something, try it, and if it doesn’t suit they will change the decision. This approach doesn’t require any commitment. But this is definitely not an efficient or effective approach in corporate IT.
8.      Prototyping?
From the test phase on, the solution is available to some business people. User Acceptance Tests (UAT) are meant to see whether the solution matches the demand. But tests are for the business stakeholders also a way to verify and to experience if it is what they needed. Although the product of the project may passes the UAT, the business may become aware of some mismatches. New change requests may be formulated. Once the solution is operational, they can verify whether the solution fits into the operational environment. In fact, to some degree, they use a finished product as a prototype, as a proof of concept. This is also a very inefficient, very costly and very risky way of working. Let’s send some of our astronauts into space to see if our rocket works.
 
To conclude
Needs and issues are detected late. A lot of time is wasted before the IT department receives the business demand. This demand is unreliable. It is only assumed that the solution described in the demand will solve the problem and not create new ones. The commitment of the business community is inexistent or, if it would exist, it would be unjustified. The IT department has to work under immense pressure and in less than optimal conditions. It lacks of resources. The IT projects need some stability and some facts they can rely on. They don't have them. The project is actually used to explore the needs, the issues, the context and the solution.
 
It is very unlikely that the IT department can ever deliver a decent quality It implementation.
And it is also unlikely that simply building what they described in each of their demand will ever satisfy the business community.
We need to rethink the relationship and the whole process.
Post “CIT 5.2” provides some additional insight in about the position of the IT department within the approach described above.