TDD saves money and developement time

there's always a problem - to show developers that TDD is actually decreasing development time, or to show management, that is saves money.

their main argument being - time spent developing tests is time wasted. why
test, when you can code and it will work.

this is very short-sighted, as in software engineering(dare i say), writing initial code is actually a lesser amount of time, spent by developer on the given feature.

i came up with this idea, to approximately calculate costs of developing a feature in the scope of the project, calculating all the costs associated with it. it's highly unscientific and not based on actual data. most numbers i'm using are just pure gut feelings, or i've read them somewhere and can't remember the source.

the goal - make you think with figures for your project and see - how much time
and money you can save with TDD.

let me explain and show some figures, that will, hopefully, be reflecting one of the common cases in the history of th feature, in the context of the project.

let's say that developer is given a specification to code a feature.

let's assume that we're 3 month into 9 months project, with 2 major refactoring coming up.

how does the cost of changing(CoC) the code(add or edit) change with the life of a product?

kent beck argues, that it's close to exponential. let's assume that. also, let's take the adjusted TDD CoC graph from Scott W. Amber and juxtaposition them.

so, let's assume that in the whole project, a given feature will invoke the following activities:
- design
- code
- bugfix
- regression
- change
- refactor

further, let's assume, that we have a 9 months to deliver a product, and we'll have major refactorings on months 7 and 11.

let's assume each refactoring will require 30% of original development time and is already reflected in graphs. without refactorings they will be climbing up faster.

let's say fixing a bug or regression would cost us 15% of feature implementation time.

and feature needs to be changed, according to beta-testers feedback. let's assume it'll take 50% of current feature implementation time.

also, let's say that in 1% of regressions, customer will be losing $1000 per hour(be it fleing customers, or inability to attract new ones), while feature isn't fixed. and it'll have to wait 10 hours to fix - you know, while someone notice, inform customer, he informs PM, write a ticket, it's 2 AM, in the morning it goes to urgent tickets and will be fixed eventually.

typical developement-testing-production feedback cycle usually consists of QA person, checking provided functionality, after dev said it's ready.and they're pretty good and spot 100% of bugs(yeah, right), introduced by developer in the first place. also, developed feature requires 10% of developement time to check.

i'm not taking into account the time spent testing feature by developer himself(when not using TDD), although it may be pretty substantial, approaching or even surpassing the time, needed to write tests.

now, the whole feature life cycle, with timings adjusted against CoC, using TDD and not.

so feature takes 10h to write. we're writing it on the third month of the project. assuming developer rate of $10/h and qa rate of $5/hour.

without TDD:
===========

initial time to write: 10*1.4 = 14h
qa check: 10*0.1 = 1h

fix bug: 10*0.15*1.5 = 2.25h
qa check: 10*0.1 = 1h

refactor: 10*0.3*1.8 = 5.4h
qa check: 10*0.1 = 1h

regression: 10*0.15*2.1 = 3.15h
qa check: 10*0.1 = 1h
adjusted customer money loss: 10*1000*0.01 = $100

feature: 10*0.5*2.6 = 13h
qa check: 10*0.1 = 1h

refactoring: 15*0.3*2.8 = 12.6h (if you dare, at all)
qa check: 10*0.1 = 1h


totals:
developer time: 50.4h
qa time: 6h
lost money: $100

total money burned on this feature is: $464 + $30 + $100 = $644


with TDD:
========
assumptions:
- regressions is tested code are rare, let's say it's 3%.
- writing tests upfront improves code. need 20% less time to refactor.
- writing tests in TDD takes 30% of feature time.
- same qa

initial time to write: 10*1.3 = 13h
tests: 13*0.3 = 3.9h
qa check: 10*0.1 = 1h

fix bug: 10*0.15*1.4 = 2.1h
qa check: 10*0.1 = 1h

refactor: 10*0.3*1.5 = 4.5h
no need to qa for regressions here

regression rate of 3% is too small to change the final picture here, but let's
calculate them just for fun.
regression: 10*0.15*1.5 = 2.25h
qa check: 10*0.1 = 1h
1% customer money loss: 10*1000*0.01 = $100

adjusted developer time: 2.25*0.03 = 4 min
3% adjusted 1% customer money loss: $100 * 0.03 = $3

feature: 10*0.5*1.6 = 8h
tests: 8*0.3 = 2.4h
qa check: 10*0.1 = 1h

refactoring: 15*0.3*1.7 = 7.65h
no need for qa


totals:
developer time: 33.9h
qa time: 3h
lost money: $3

total burned money: $339 + $15 + $3 = $357




conclusions?

TDD saves money and developement time. of course total number vary greatly, but
i'm sure that, with real world data, TDD superiority will still hold.

there are other TDD benefits, not reflected in the picture, i.e. using TDD will
require less time to refactor, because more thinking and design is put into code
upfront.

and it's only on 11 months timeframe average project. make it two years, or
increase customer money loss on regressions - and advantages really stand out.


links:
Examining the Agile Cost of Change Curve

python typecheck with decorators

yesterday i was thinking on the alternative to the exceptions.

like, you know, something that would safely propagate through functions without having to wrap up every damn function call in try-except block, allowing only topmost code to react on errors.

what i had in mind was something like Maybe monad in Haskell, where function could return a value, or nothing, and that would get accepted as a correct return value by other operators or functions.

of course, that would require adding Haskell type system to python :)

speaking of which, i found the idea of type checking really interesting, so i came up with this dynamic type check decorator. i think it's a nice step forward towards correct code. easy on eyes, small, allows custom error handling and reporting.

# exmaple of checking inputs and output of a function
def signature(*types):
    def check_inputs(f):

        acc_types = types[:-1]
        ret_type = types[-1]

        assert len(acc_types) == f.func_code.co_argcount

        def new_f(*args, **kwds):
            for (a, t) in zip(args[:-1], acc_types):
                assert isinstance(a, t), \
                       "arg %r does not match %s" % (a,t)

            result = f(*args, **kwds)
            assert isinstance(result, ret_type), \
                   "return value %r does not match %s" % (result,ret_type)

            return result

        new_f.func_name = f.func_name
        return new_f

    return check_inputs

# last argument is a type of the return value
@signature(int, int, list, bool, str)
def report(a,b,c,d):
    number =  a+b+sum(c)

    if d == False:
        return str(False)

    return str(number)

print report(1,2,[1,2,3],False)

php considered immature

so i'm writing this mailing sender in php. test driven design works btw. just need to get used to it. anyway, over a week i ran into 3 behaviors, that really made me scratc hmy head a bit:

1) i installed Pear::Mail, and skipped dependencies. "well", i thought, "it'll tell me if i'll need any". so i'm creating html emails, building whole test suite, code, class etc. everything works except one minor detail. when it should actually send email - it does nothing. silently exit, without sending. no errors, nothing. turns out - it needed Pear::Mail#net_smtp. took me some hours to figure out.

2) the plan was to spawn a process with proc_open(), send info to it through pipe and get an output from pipe. easy, right? but! what happens, if the process hangs? just poke it until predefined timeout occurs. yeah, not that easy. turns out proc_open() pipes can't be set to non-blocking, meaning if parent tries to read from process and process hangs - parent hangs as well. now i have to use shared memory or whatever, just because those pipes can't be set to non-blocking.

3) the script had 3 include files and was spawned by parent script. when parent was run from command line - all went ok. when parent was called from apache - it didn't do what expected. after some poking i figured out, that it didn't return from the second include. nope. just entered it, ran through it and exited, never returning to parent to include third script and, actually, do what's needed. this one really took some time. turned out - it was whitespace, after the ending tag "?>". how on earth would that be relevant? no idea. but it worked from webserver then, as well.

three different bad behaviors, over the course of a week. talk about language maturity.

productivity

last 2 weeks i was working on a project started anew, so i decided to make a short summary on it. the only metric i could think of is LOC, so i went with it. here we go, numbers represent lines/words/characters in each dir:

classes 665 1857 20334

js 228 469 6586

config 18 45 625

templates 336 688 10849

views 390 746 9295

css 216 368 3398

unittests 498 1293 15338

total:
php: 1073
templates: 336
js: 228
css: 216
unittests: 498

grand total: 2351
time: 2 weeks ~ 40*2 = 80 hours
productivity: 2351/80 = 29.39 lines per hour

so the productivity is very close to standard 30 lines/hour. guess i'm doing okay :)

smalltalk

wow, that's one cool explanation on class methods :)

When you create a class, Smalltalk creates a metaclass; and just like a class describes how methods for its instances work, a metaclass describes how class methods for that same class work.

Set is an instance of the metaclass, so when you invoke the #new class method, you can also say you are invoking an instance method implemented by Set class. Simply put, class methods are a lie: they're simply [metaclass] instance methods that are understood by instances of metaclasses.

another job well done!

phew, finished the haskell book. solid stuff. will be looking to do some projects in it(parser? compiler?).

moving to flex meanwhile. need to check what it's all about. ecmascript is kinda fun - dynamic, prototyped language. too bad that i suck at design :)

haskell power!

i'm still reading a book on haskell - "Haskell - The Craft of Functional Programming".

nice introductory and overall stuff.

so, haskell have a lot of high level functions(what a surprise for a functional programming language!), today two of them impressed me.

1) iterate: "creates an infinite list where the first item is calculated by applying the function on the secod argument, the second item by applying the function on the previous result and so on".

Input: take 10 (iterate (2*) 1)

[1,2,4,8,16,32,64,128,256,512] ('take' takes first n items of a list)

so it takes an argument as a starting point and continues applying function to it.

it can take lambda functions, like this:

Input: take 10 (iterate (\x -> (x+3)*2) 1)

[1,8,22,50,106,218,442,890,1786,3578]

and i thought that this can be used to calculate fibonacci numbers!

take 10 (iterate (\(x,y)->(y,x+y)) (1,1))

[(1,1),(1,2),(2,3),(3,5),(5,8),(8,13),(13,21),(21,34),(34,55),(55,89)]

2) list comprehensions. not exactly a function, but very nice construct. they may return infinite lists.

it should be pretty much familiar to python programmers, though more powerful.

[ f x | p1, p2, p3... ]

each value can be transformed by function f
p1, p2... - conditions that evaluate to boolean OR generators

"so what" you may say, nothing new. yeah, except p1, p2... can use values from generators on the left!

e.g., we can have a infinite list of pythagorean triples:

[ (x,y,z) | z <- [2..], y <- [2..z-1], x <- [2..y-1], x^2 + y^2 == z^2 ]

[(3,4,5),(6,8,10),(5,12,13),(9,12,15),(8,15,17),(12,16,20),(15,20,25),(7,24,25),(10,24,26),(20,21,29)]

i was really excited about those two. enough to blog about 'em :)