Software Test estimation is a management activity which approximates how long a Task would take to complete. Estimating effort for the test is one of the major and important tasks in Test Management.
This topic is a mixture of practical experiences and estimation theory. The purpose of this topic that the Test Leads, Managers or aspiring leads, managers must aware of all the test estimation techniques.
Why test estimation?
Two questions you can expect from your clients when discussing potential test engagements are
- How long will this testing take?
- How much will it cost?
Estimation is done often because it is expected to help in predicting how much will the project cost and when will the project get completed. Proper analysis and effort estimation is necessary for successfully planning for a testing project. Any flaw in critical estimation phase, results in missing the project deadlines, reduces ROI and loses of customer’s faith. Remember – Bad estimation can lead to poor distribution of work.
Estimate of What?
- Resources: Resources are required to carry out any project tasks. They can be people, equipment, facilities, funding, or anything else capable of definition required for the completion of a project activity
- Time: Time is the most valuable resource in a project. Every project has a deadline to delivery
- Human Skills: Human skills mean the knowledge and the experience of the Team members. They affect to your estimation. For example, a team, whose members have low testing skills, will take more time to finish the project than the one which has high testing skills
- Cost: Cost is the project budget. Generally speaking, it means how much money it takes to finish the project
The popular test estimation techniques
- FIA (finger in the air) or best guess
2. Ad-hoc method
3. Experience Based – Analogies and experts
5. Delphi technique
6. Three-point estimation (successive calculation)
7. Function points / Test point Analysis
8. Percentage of development effort method
9. Percentage distribution
10. Use case point estimation method
Let’s discuss one by one:
1. Best Guess – This technique is purely guesswork and based on the some sort of experience.
The method is very common, but since it is based on your gut feeling, its uncertainty contingency is probably around 200% or even higher
- Ad-hoc method – The test efforts are based on tentative timeframe. The timeline set by managerial or marketing personnel or by client without any guess/experience. Alternatively, it is done until the budgeted finances run out.
This is very common practice in extremely immature organizations and has error margins of over 100% at times
- Experience Based – Analogies and experts:
- Metrics collected from previous tests
- You already tested similar application in previous project
- Inputs are taken from Subject Matter experts who know the application (as well as testing) very well
- Work Breakdown Structure – It is created by breaking down the test project into small pieces. Modules are divided into sub-modules. Sub modules are further divided into functionalities and functionalities are divided in sub-functionalities.
Review all the requirements from Requirement Document to make sure they are added in WBS. Now you figure out the number of tasks your team needs to complete. Estimate the duration of each task
- Delphi technique – Same as above WBS. Here functionalities and each task is allocated to each team member. Then team member gives estimate that they will take this much hours to complete the task.
Averagely, this technique gives good confidence in the estimation. This technique can be combined with other techniques
- Three-point estimation – This technique is based on statistical methods in this technique, task is broken down into subtasks (similar to WBS) and then three types on estimation are done on each chunk
- Optimistic Estimate (Best case scenario in which nothing goes wrong and all conditions are optimal.) = a
- Most Likely Estimate (most likely duration and there may be some problem but most of the things will go right.) = m
- Pessimistic Estimate (worst case scenario which everything goes wrong.) = b
Formula to find Value for Estimate (E) = a + (4*m) + b / 6
Standard Deviation (SD) = (b – a)/6
- Function Point/Testing Point Analysis: The FP technique is a direct indicator of the functionality of software application from the user’s perspective. This is the most accepted technique used to estimate the size of a software project
The whole project task into small task by using WBS method. Now you estimate the size of those tasks. Let’s practice with a particular task “Create the test specification”
The size of this task depends on the functional size of the system under test. The functional size reflects the amount of functionality that is relevant to the user. The more number of functionality, the more complex system is.
Prior to start actual estimating tasks effort, functional points are divided into three groups like Complex, Medium Simple
Based on the complex of software functions, the Test Manger has to give weightage to each functional point.
After classifying the complexity of the function points, you have to estimate the duration to test them. Duration means how much time needs to finish the task.
- Total Effort: The effort to completely test all the functions of the website
- Total Function Points: Total modules of the website
- Estimate defined per Function Points: The average effort to complete one function points this value depends on the productivity of the member who will take in charge this task.
Capers Jones basic formula:
Number of Test cases = [Number of Function Points] x 1.2
Total Actual Effort, TAE = (Number of Test cases) * (Percentage of development effort /100)
This method is done in a case when a detailed low level design document or requirement document is available (i.e. measure of function point is available) & Previous data for development and testing is available. But now days, when we are using agile and iterative methodologies to deliver projects, so most of the times all this documentation is not available.
- Percentage of development effort method
Here the assumption is that a more complex business application may require more testing effort. The test effort required is a direct proportionate or percentage of the development effort.Note: The development effort can be estimated using line of code (LOC) or function point (FP) which is not in the our scope.
If a previous project with 500 FPs required 50 man hours for testing, the percentage of testing effort is calculated as:
P = (50 / 500) * 100 =10%
For the current project with a development effort, say 1500 FPs, the testing effort is:
Total Actual Effort, TAE = 1500 * (P/100) = 1500 * (10/100) = 150 man hours.9. Percentage distribution – Here all the phases of SDLC are divided in parts and assigned effort in %. Like –
Project management 7%
Test (all test phases) 27%
Installation and training 6%
- Use case point estimation method: Use case point (UCP) method is gaining popularity because now-a-days application development is modelled around use case specification. The test case development is normally kicked off after baseline use case. So the various factors in use case give a direct proportion to the testing effort.
Use case is a document which well specifies different users, systems or other stakeholders interacting with the concerned application. They are named as ‘Actors’. The interactions accomplish some defined goals protecting the interest of all stakeholders through different behavior or flow termed as scenarios.UCP Estimation Method in brief
1. Obtain unadjusted actor weight (UAW)
2. Determine unadjusted use case weight (UUCW)
3. Calculate unadjusted use case points (UUCP) UUCP = UAW + UUCW
4. Determine the technical/environmental factor (TEF)
5. Compute the adjusted use case point (AUCP) AUCP = UUCP * [0.65 + (0.01 * TEF)]
6. Arrive at final effort using a conversion factor Total Actual Effort = AUCP * CF
Test estimation best practices
This topic introduces general tips on how to estimate Testing accuracy.
- Add some buffer time: Many unpredictable things may happen to your project, such as a talented team member quits his job suddenly, the testing takes more time than estimated to complete… etc. That why you need include some buffer in your estimation. Having a buffer in the estimation enables to cope for any delays that may occur.
- Account Resource planning in estimation: What should you do if some members in your team take long leaves? It may delay the project. Resource planning in estimation plays a key role. The availability of resources will help to make sure that the estimations are realistic. Here you have to consider the leaves for your team member, generally long leaves.
- Use the past experience as reference: Experiences from past projects play a vital role while preparing the time estimates. Because some project may be some similarity, you can reuse the past estimation. For example, if you use to do a project like testing a website, you can learn from that experience, try to avoid all the difficulties or issues that were faced in past projects.
- Stick to your estimation: Estimation is just estimate because it may go wrong. In early stages of the project, you should frequently re-check the test estimations and make modification if needed. We should not extend the estimation after we fix it, unless there are major changes in requirement, or you have to negotiate with customer about the re-estimation