their main argument being - time spent developing tests is time wasted. why
test, when you can code and it will work.
this is very short-sighted, as in software engineering(dare i say), writing initial code is actually a lesser amount of time, spent by developer on the given feature.
i came up with this idea, to approximately calculate costs of developing a feature in the scope of the project, calculating all the costs associated with it. it's highly unscientific and not based on actual data. most numbers i'm using are just pure gut feelings, or i've read them somewhere and can't remember the source.
the goal - make you think with figures for your project and see - how much time
and money you can save with TDD.
let me explain and show some figures, that will, hopefully, be reflecting one of the common cases in the history of th feature, in the context of the project.
let's say that developer is given a specification to code a feature.
let's assume that we're 3 month into 9 months project, with 2 major refactoring coming up.
how does the cost of changing(CoC) the code(add or edit) change with the life of a product?
kent beck argues, that it's close to exponential. let's assume that. also, let's take the adjusted TDD CoC graph from Scott W. Amber and juxtaposition them.
so, let's assume that in the whole project, a given feature will invoke the following activities:
- design
- code
- bugfix
- regression
- change
- refactor
further, let's assume, that we have a 9 months to deliver a product, and we'll have major refactorings on months 7 and 11.
let's assume each refactoring will require 30% of original development time and is already reflected in graphs. without refactorings they will be climbing up faster.
let's say fixing a bug or regression would cost us 15% of feature implementation time.
and feature needs to be changed, according to beta-testers feedback. let's assume it'll take 50% of current feature implementation time.
also, let's say that in 1% of regressions, customer will be losing $1000 per hour(be it fleing customers, or inability to attract new ones), while feature isn't fixed. and it'll have to wait 10 hours to fix - you know, while someone notice, inform customer, he informs PM, write a ticket, it's 2 AM, in the morning it goes to urgent tickets and will be fixed eventually.
typical developement-testing-production feedback cycle usually consists of QA person, checking provided functionality, after dev said it's ready.and they're pretty good and spot 100% of bugs(yeah, right), introduced by developer in the first place. also, developed feature requires 10% of developement time to check.
i'm not taking into account the time spent testing feature by developer himself(when not using TDD), although it may be pretty substantial, approaching or even surpassing the time, needed to write tests.
now, the whole feature life cycle, with timings adjusted against CoC, using TDD and not.
so feature takes 10h to write. we're writing it on the third month of the project. assuming developer rate of $10/h and qa rate of $5/hour.
without TDD:
===========
initial time to write: 10*1.4 = 14h
qa check: 10*0.1 = 1h
fix bug: 10*0.15*1.5 = 2.25h
qa check: 10*0.1 = 1h
refactor: 10*0.3*1.8 = 5.4h
qa check: 10*0.1 = 1h
regression: 10*0.15*2.1 = 3.15h
qa check: 10*0.1 = 1h
adjusted customer money loss: 10*1000*0.01 = $100
feature: 10*0.5*2.6 = 13h
qa check: 10*0.1 = 1h
refactoring: 15*0.3*2.8 = 12.6h (if you dare, at all)
qa check: 10*0.1 = 1h
totals:
developer time: 50.4h
qa time: 6h
lost money: $100
total money burned on this feature is: $464 + $30 + $100 = $644
with TDD:
========
assumptions:
- regressions is tested code are rare, let's say it's 3%.
- writing tests upfront improves code. need 20% less time to refactor.
- writing tests in TDD takes 30% of feature time.
- same qa
initial time to write: 10*1.3 = 13h
tests: 13*0.3 = 3.9h
qa check: 10*0.1 = 1h
fix bug: 10*0.15*1.4 = 2.1h
qa check: 10*0.1 = 1h
refactor: 10*0.3*1.5 = 4.5h
no need to qa for regressions here
regression rate of 3% is too small to change the final picture here, but let's
calculate them just for fun.
regression: 10*0.15*1.5 = 2.25h
qa check: 10*0.1 = 1h
1% customer money loss: 10*1000*0.01 = $100
adjusted developer time: 2.25*0.03 = 4 min
3% adjusted 1% customer money loss: $100 * 0.03 = $3
feature: 10*0.5*1.6 = 8h
tests: 8*0.3 = 2.4h
qa check: 10*0.1 = 1h
refactoring: 15*0.3*1.7 = 7.65h
no need for qa
totals:
developer time: 33.9h
qa time: 3h
lost money: $3
total burned money: $339 + $15 + $3 = $357
conclusions?
TDD saves money and developement time. of course total number vary greatly, but
i'm sure that, with real world data, TDD superiority will still hold.
there are other TDD benefits, not reflected in the picture, i.e. using TDD will
require less time to refactor, because more thinking and design is put into code
upfront.
and it's only on 11 months timeframe average project. make it two years, or
increase customer money loss on regressions - and advantages really stand out.
links:
Examining the Agile Cost of Change Curve
No comments:
Post a Comment