Agile process

1.       Agile Definition

When developing software using the Agile methodology:

1. The customer creates stories that describe the functionality of the software.
1. These stories are small units of functionality taking about a week or two to code and test.
2. Programmers provide estimates for the stories
3. The customer decides, based on value and cost, which stories to do first.
4. Development is done iteratively and incrementally
1. Each two weeks, the programming team delivers working stories to the customer
2. Then the customer chooses another two weeks worth of work
3. The system grows in functionality, piece by piece, steered by the customer. This means that progress is measured and tracked based on the observable behavior of the system, rather than judging design artifacts such as SDS, TSD etc.
   
   

Agile/XP relies on evolutionary design techniques to keep the software design good while adding new functionality. Given how fluid software requirements can be, evolutionary design is a critical practice for any software project. The trouble is that traditional development practices do not provide good techniques for safely evolving designs as requirements mutate.

2        Roles in Process

• Coach / Tracker – This role is truly a project manager.  This person is responsible for the following:
• Facilitating planning game
• Managing the releases
• Overall progress of the project
• What the individual team members are working on
• Giving project status to those within the project and outside the project
• Keeping the project tracking tool up to date
• Facilitating the daily standup
• Business representative
• Involvement in planning game
• Prioritize business requirements
• Be available on daily basis to answer questions
• Constantly reviewing the project, giving feedback at the earliest possible time
• Analyst
• Involvement in planning game
• Analysis for project / iteration / user stories / tasks
• Entering / maintaining user stories in Agile tool
• Work with Tester in review and execution of SIT cases
• Developer
• Following the agile / test driven processes
• development
• Involvement in planning game
• Tester (Q/A)
• Involvement in planning game
• Writing SIT cases
• Overseeing execution of SIT cases
• Managing testing effort across SIT / SAT / UAT
• Provide SIT status
• Track implementation steps for project
• Establish / maintain traceability between user stories and SIT cases
• UAT
• Involvement in planning game
• Writing UAT cases
• Execution of UAT cases

3        Project Process

3.1       Pre-Project

Before the project starts all the necessary resources need to be allocated and dedicated to the project.  Starting projects before the dedicated resources are all available will lead to problems in the long run.   The following list outlines what is necessary to get a project started:

• Resources
• Fully dedicated resources for the following roles:
• Coach / Tracker
• Analyst
• Developer
• SIT
• Resources that aren’t fully dedicated
• Business Representatives
• Architect
• UAT
• Optional resources
• Database Engineer
• We are currently using the following formula for the number of Analyst / Developer / Tester – 1 Analyst / 4 Developers / 1.5 Testers.  This is not set in stone and will change from project to project.
• Project room or area – the project team needs to be working together in a dedicated space where the team members will spend the majority of their time.  There are times when individual members need to leave the room/area, but this should be kept to minimum.
• Project Planning game

3.2       Project Planning Game

There should be planning game meetings that are for the overall project, these should discuss the overall requirements of the project.  The initial planning game’s purpose is to lay out the high level requirements for the project and to give an initial priority to the requirements.  These priorities will aid in laying out what will be done in each of the releases, and then each of the iterations that lead to a release.

The team needs to consider what they want to get out of each release to make sure that the iterations leading up to the releases are focused on the appropriate work.  While the release may be a significant driver towards what is being done in the iterations, the overall project must also be considered and the development should be aimed at finishing the work for the overall project as well as releasing logical groups of functionality during the releases.

This meeting should be attended by the Coach / Tracker, Tester, Analyst and a developer should be in these meetings.

3.3       Assumptions

• Most of the development will be done using Pair Programming.
• The entire process is team driven.  The Coach / Tracker needs to be moving the project
• Business is truly part of the team, there is a need to have

4        Working Releases

Planning for the release starts with the project planning game.  This is where the initial functionality will be initially laid out to hit individual releases.  Each iteration planning game should address how the release will be affected as well as what is being done for the current iteration.

• Project Planning Game
• Iterations
• Planning game
• SIT case creation / modification and execution
• SIT – This will be a 1 week period dedicated to test execution following the end of the last scheduled iteration and prior to SAT start.  All functional coding and SIT cases should be completed when SIT starts.  This is also the point when user requirements will no longer be accepted.
• SAT/UAT – This will be the 3 week period prior to the release date.  This is controlled outside of the agile process. 
• Post Release – This usually lasts for the 2 weeks following the release, if any problems arise the project team would need to be involved to resolve the issues.

The entire Technology team (Analyst, Developers, Testers) should be dedicated to the SIT phase to fix defects, help the testers execute SIT cases and participate in exploratory testing.  At least one pair of developers and one tester need to be assigned to the SAT phase of the project. 

The next iteration should be starting as the current release starts SAT.   This iteration will be working towards the next release, but the team needs to understand that the current release is the main priority.  It is important to set expectations that the team will be moving forward with the next iteration, but any or all of the resources on the new iteration may be pulled into the SAT phase or post-release if the need arises.

5        Working within Iterations

5.1       Planning Game

o    Coach / Tracker needs to determine the ideal hours that are available for the iteration.

§  First step is to determine the total hours available to the iteration, by putting everyone in at 40 hours per week, and then subtracting the vacation time and training time.

§  Once the available time is determined the ideal hours available can be calculated by multiplying the velocity by the actual.  The velocity is something that may change over time, currently we are using 75%.

§  An alternative is to just take the number of ideal hours that were completed in the last iteration or average of last iterations and use that as your benchmark.

§  This is purely an estimate to give the Tracker an idea of how many hours to plan for in the coming iteration, it is a meaningless number outside of this purpose.

o    Discuss stories as a team

§  Team should include the Technology Team (Analysts, Developers, Testers & Coach) along with the business partners

§  It is crucial to work out as much information as possible about the user stories so that the estimating that will come after is as accurate as it can be.

o    Have dedicated estimating as a Technology team

§  The entire technology team needs to be in this meeting and it should be outside of the normal work area to limit distractions.

§  Since everyone has heard the requirements from the users it should be a discussion about how to break the story into tasks and how long each task should take

§  Each story should have an estimate for how long it will take in ideal hours to Analyze, Code and Test.

·         Ideal Hours are defined by working in a vacuum, no distractions nothing else going on, just being able to work without outside issues.

·         These estimates should be on how many hours it will take 2 developers together to do it.  If for example it should take 4 hours for the 2 developers doing pair programming then the estimate for development would be 8 hours.

·         User stories should not be more than an average of a few days worth of work.  If there are a total of 2 weeks worth of work or more the story needs to be broken down to a more manageable size.

·         Tasks should have an average size of no more than one day.  If an individual task has a total of 2 days or more then the task needs to be broken down to a more manageable size.

§  Consider testability of stories.  If several stories are required to be completed for a function to be SIT tested, group these stories together and provide a technology priority within the function. The business can prioritize the function within an iteration but the stories identified must be completed as a group.

o    Once the estimates are complete the coach / tracker needs to re-prioritize the user stories with the business

§  The business’ priorities may change once the estimates have been completed.  A story may be a high priority if it takes 10 hours, but not if it takes 40 hours.

§  If there are user stories that have not been estimated yet and the business wants to evaluate them the coach would then ask the team to estimate only the stories that the business realistically thinks will end up in the current iteration.  If these stories were not discussed in a previous planning game, a short discussion on the story should take place prior to estimation between the Technology team and the business partners

5.2       Stand Up Meeting

There needs to be daily standup meetings within any agile project.  The stand up meetings should be at a set time of day every day, if it is possible; keep it consistent.  The meeting should be facilitated by the Coach / Mentor.  If the coach is not available there should be a known stand-in to take over facilitating the conversation.  The meeting should be limited to 15 minutes; people are encouraged to stand up during the meeting to keep it short.

The main purpose of the stand-up is to get everyone on the project up to date with what everyone else is doing.  Each person on the team will talk about what they have worked on since the last stand-up and what they plan on working before the next meeting.  This is a good opportunity for people to ask questions and to make helpful suggestions.

5.3       Unit Acceptance Tests

The SIT and Analyst resources assigned to the project are responsible for gathering and documenting the Unit acceptance tests (user cases) with the business partner.  These tests should be segregated by user story, allowing them to be applied when the individual user stories have been completed.  The Unit Acceptance Tests will be handed off to the developers for automation and execution.  Once the Unit Acceptance Test has passed, the code may be checked in

5.4       Development

o    Update user stories through tool at the end of the day or whenever an user story is completed

§  Accuracy of time is extremely important

§  Time needs to be entered for development, analysis and test

o    Pairing

• This follows the pair programming laid out in the XP book
• There should be goal of keeping pairing around 75% of the time
• Pairing should be promiscuous, in that our pairs are not set. 
• Every day pairs should evaluate if they will stay together or switch round
• If a pair is still working on an user story the only rule is that one of the members must remain on the user story, two new developers cannot start working on an existing story.
• Breaking up a pair during a user story should not be discouraged; this allows more developers to be involved and the knowledge of how to work on that area will be spread.   Also there will be another set of eyes on the code that may catch problems or issues with the process being followed.

o    Follow standards / process

o    How do we test earlier in the iteration?

o    Defects are assigned through inform throughout the project

5.5       Feedback from Business Representatives

Feedback from the business representatives needs to be gathered continuously.  Whenever an user story is completed the business needs to give their feedback on how the works meets their expectations.  This feedback should be gathered after the development has gone through the user acceptance tests. Along with the business, the tester should also be involved.

Done properly the work will be in a constant state of approval from the business users on functionality and form.  There will be need for actual testing by the user groups but the functionality coded should never be in question.  Beyond

5.6       Ongoing UAT

UAT testing is outside of the normal business representatives giving their approval of the product. 

5.7       Ongoing SIT

o    SIT Process

§  Participate in Planning games and estimation sessions for the iteration

§  During the first 2 weeks of the first iteration of a project

• Gather environment requirements and forward to environment services
• Identify Test Deliverables, provide estimates for test execution of deliverables and assign test case creation resources for all functions identified in the project planning game. 
• Begin writing SIT cases for functions discussed in the 1^st iteration Planning game

§  By week 3 of the iteration you should receive your first build.  If the SIT environment is not available, begin execution in the dev environment.  Testing should be happening in week 3 & 4 of the current iteration and week 1 & 2 of the next iteration.

§  During week 1&2 of iteration 2, you should be splitting your time between execution of tests from iteration 1 and SIT case creation for iteration 2.

§  Throughout the project, the SIT lead is responsible for tracking all setps that will be required to implement the project into production including, builds, schema, scripts, standalone deployments, etc.  Along with this, a list of dependencies on other projects and projects that are dependent on your project should be kept.

§  There is a 2 week SIT window following the end of the last iteration for the project to execute all remaining SIT cases, perform defect fixes, complete regression and perform exploratory testing.  All members of the Technology team will be participating.   Prior to this SIT window, all SIT cases should be documented in Quality Center and have been reviewed by the analyst and developer assigned to the function.

§  Review the functionality being planned for release with the SAT and UAT teams.  Coordinate production validation with UAT and provide any further assistance as needed.

§  Regression will also be required in the SAT environment if code merges or new builds are delivered for any component of the integrated environment that Portal uses.

§   

5.8       Weekly review meetings

The weekly review meeting is meant to insure that all teams are going through the test driven development process and agile process during the iteration.  This meeting needs to take place on Fridays after lunch.  A manager needs to facilitate going through the user stories that were completed this week.  It is imperative to make sure that the developers have stuck to the test driven model that is laid out in the TDD document.  If  a pair hasn’t followed the model and are missing test cases or there is another lapse in the process then they need to get this fixed by the stand up on Monday morning.

5.9       Wrap-up / review iteration meeting

This is basically a post mortem on the iteration.  This can cover agile, test driven or actual project issues and discussions.

6        Project Management

6.1       Project Management Tool

6.2       Weekly Status Reports

Status reports should be done at the beginning of every week, after the hours for the previous week have been updated.  The status report should contain the following information:

• Summary of the week
• Accomplishments
• Detail of any issues
• Plans for upcoming week
• What is currently known to be coming up
• Any milestones or issues that are about to happen
• User Stories
• All the user stories that are being considered for this iteration should be shown on the report
• There should be designation if the user story will be considered in the iteration or if it will be done if there is time
• The date that the user story was completed
• Resource Utilization
• This is based on the number of hours that were entered into the project management tool
• SIT Progress
• Functional Areas planned to be tested, estimate of effort for execution, assignments for test case writing and execution, planned and actual dates for cases and execution and defect statistics
• Issues
• A detailed list of the issues that are currently open including:
• Date opened
• Issue Description
• Owner of Issue
• Estimated date fixed
• Closed date
• The Issue should be listed on the status report until it is fixed and the fact that it is closed shows on one status report

 

7        Process Definition

The purpose of this section is to define a set of ideal practices for an agile software development project:

7.1       Agile Techniques

Agile is a collection of best practices

Customer Team Member – Teams have someone (or a group of people) representing the interests of the customer. They decide what is in the product and what is not in the product.

Planning Game - XP is an iterative development process. In the planning game, the customer and the programmers determine the scope of the next release. Programmers estimating the feature costs. Customers select features and package the development of those features into small iterations (typically 2 weeks). Iterations are combined into meaningful end user releases.

User Story – A User Story represents a feature of the system. The customer writes the story on a note card. Stories are small. The estimate to complete a story is limited to no greater than what one person could complete within a single iteration.

Small Releases – Programmers build the system in small releases defined. An iteration is typically two weeks. A release is a group of iterations that provide valuable features to the users of the system.

Unit Acceptance Testing – The SIT and analyst resources writes unit acceptance tests with the business partner. The tests demonstrate that the story is complete. The programmers automate and execute unit acceptance tests prior to checking in their code.

Open Workspace – To facilitate communications the team works in an open workspace with all the people and equipment easily accessible.

Test Driven Design – Programmers write software in very small verifiable steps. First, write a small test. Then we write enough code to satisfy the test. Then another test is written, and so on. – See Test Driven Development Document

Testing is conducted as follows:

• Using Test Driven development
• Each module developed should be testable and tests written for it
• Developer writes unit tests from within an automated unit testing framework (JUnit)
• Testing framework provides test coverage information. Coverage targets are established. Each module’s tests must at least reach the target before being checked in.

Metaphor – The system metaphor provides an idea or a model for the system. It provides a context for naming things in the software, making the software communicate to the programmers.

Simple Design – The design in XP is kept as simple as possible for the current set of implemented stories. Programmers don’t build frameworks and infrastructure for the features that might be coming.

Refactoring – As programmers add new features to the project, the design may start to get messy. If this continues, the design will deteriorate. Refactoring is the process of keeping the design clean incrementally. Performed:

• During (Test Driven) development
• During bug fixes
• When the developer recognizes a code smell(Appendix A)

Continuous Integration – Programmers integrate and test the software many times a day. Big code branches and merges are avoided by provision of the following:

• The project should have an automated build, an automated deployment and automated testing
• A new build and test should be performed on a build server periodically
• The results should be reported to every team member and it should be an established team practice to immediately fix the build
• A working build is everyone's top priority

Collective Ownership – The team owns the code. Programmer pairs modify any piece of code they need to. Extensive unit tests help protect the team from coding mistakes.

Coding Standards – The code needs to have a common style to facilitate communication between programmers. The team owns the code; the team owns the coding style.

Pair Programming – Two programmers collaborate to solve one problem. Programming is not a spectator sport.

Sustainable Pace –The team needs to stay fresh to effectively produce software. One way to make sure the team makes many mistakes is to have them work a lot of overtime

The following practices should also be considered:

Source Control:

• All code, non-code documentation, build scripts, database schema, data scripts, and tests should be stored in Clearcase
• Code should not be checked into the build until it compiles, has tests and is passing its tests with the prerequisite coverage.

Work-In-Progress:

• A working version of the latest iteration should always be available for customer feedback
• Customers can quickly identify deviation from requirements
• Shortening this feedback loop decreases the cost of change

Feedback:

• There should be a defined mechanism for project team members, including the customer, to provide feedback on the projects processes
• Hold a meeting at the end of each iteration to update/refine the process

 

A.1 Agile Checklist

Practice 1 - Refactoring:

• During (Test Driven) development
• During bug fixes
• When the developer recognizes a code smell

Practice 2 - Testing:

• Using Test Driven development
• Each module developed should be testable and tests written for it
• Developer writes unit tests from within an automated unit testing framework (JUnit)
• Testing framework provides test coverage information. Coverage targets are established. Each module’s tests must at least reach the target before being checked in.

Practice 3 – Automated Build and Deployment:

• The project should have an automated build, an automated deployment and automated testing

Practice 4 – Continuous Integration:

• A new build and test should be performed on a build server periodically
• The results should be reported to every team member and it should be an established team practice to immediately fix the build
• A working build is everyone's top priority
• Depends on Practice 3 above

Practice 5 – Source Control:

• All code, non-code documentation, build scripts, database schema, data scripts, and tests should be stored in Clearcase
• Code should not be checked into the build until it compiles, has tests and is passing its tests with the prerequisite coverage.

Practice 6 – Test Tracking:

• There should be a defined technique for recording and prioritizing development tasks and bugs
• The system should make it possible to assign responsibility for tasks to individuals.
• If tasks are tracked against estimates then the estimate should be performed by the person who will do the task

Practice 7 – Self Documenting Code:

• Code comments should be subject to the same quality requirements as the code itself

Practice 8 – Work-In-Progress:

• A working version of the latest iteration should always be available for customer feedback
• Customers can quickly identify deviation from requirements
• Shortening this feedback loop decreases the cost of change

Practice 9 – Feedback:

• There should be a defined mechanism for project team members, including the customer, to provide feedback on the projects processes
• Hold a meeting at the end of each iteration to update/refine the process