Bits & pieces on software as we are beginning to know it powered by Extreme Programming

TDR - Test Driven Review

2011-10-20T23:22:00.000-07:00

We all did it a couple of times, but it wasn't until recently that I found out how useful and efficient it was. Our fresh hires start working on production code relatively soon after their first day. Nevertheless we want to review their work before check-in.
Getting up to speed
I always start with, "Let me see the tests". It is only partially to emphasise the importance of tests, the real reason is that for me it's easier and quicker to engage in the task he has just solved. And this gives me the chance to profoundly review the code itself and how it looks.
Defining
Sometimes we don't even continue to the code, because it turns out that he misinterpreted the task at hand, and solved something else. But that's okay too, for through the tests I can very easily and clearly explain what the real task is. If we were to do this via the implementation, it would be less unambiguous.
Demonstrating
Finally, I can show him couple of coding tricks, again without being too superficial, yet being quite quick. What I mean is I can quickly refactor his code on the spot, without saying things like "... and you can imagine the rest" - no, at the end of the demonstration we have a still working code(checked by the help of the tests), but with me already having made my points on coding as we know it: expressive and duplication free.

Really.Only.Very.Rarely.

2010-09-03T11:35:00.000-07:00

The jawbreaker above is one of the most efficient half-sentence that a client could unwittingly ruin your day with. And usually alongside your day goes most of the model you have been building up. On top of all there is this huge chasm between the thinking of the customer("Really, it's no biggie. We only have like two cases a month where this and that..") and the cruel reality of software's unforgiving nature, i.e. if you want the software handle those two-times-a-month exceptions then that piece of the software needs to be developed just as well as another piece that handles all non-exceptional cases.

Laymen have a hard time understanding it, and for us it sure feels like trying to explain the colors to a blind man. No wonder, I mean we have learned software in school for years, have been around in the industry for even more. We spend most of our time in an environment where this unforgiving nature of software is self-explanatory and taken for granted.

It hasn't been till only recently that I realised how this problem is actually an opportunity. Why this jawbreaker is not something you should loath and fear -- on the contrary. This is something that can help you deliver quality software on time.

The answer is almost here. I just need you to take a second and find out on your own.

Agile in hostile environment

2010-06-14T12:28:00.000-07:00

It can be tricky to apply agile methods in a fundamentally waterfall environment. So tricky in fact, that we might be better off without agility.

Agile Manifesto was created to help creating software of value, which - according to the founders - required a different approach and a different set of methods altogether. The neccessary shift of approach was expressed in nice and clean preferences from wich the methods are more or less directly derived. If because of the circumstances we cannot make the shift in the approach, using the methods stubbornly makes little to no sense. I start with the easy ones, see if I can ruin it all for you..

(1) If the deadlines and the scope is fixed, moreover you need to deliver certain results (documentation, software, test results, etc) at certain times, you are following a plan, instead of responding to change.

(2) If you need to provide detailed req.spec, estimations, design documents, etc, then obviously working software is only of second priority, since the signing and the delivery of the contract relies heavily on these documents, rather than on working software itself.

(3) If you have to have the req.spec and the corresponding estimations upfront, you and your client - forced by contract - are bound to them(i.e. requirements and more or less estimations too). And then with even the best intentions you can get your client involved in the preliminary discussion only which are theoretic almost by definition.

(4)I left the toughest one for last.
The sobering truth is that - and a sick plot twist is around the corner, you better watch out - in such a hostile, anti-agile environment, as the above depicted one - applying agile tools and processes, in fact might be the wrong choice, and for the wrong reasons too. Because this in itself goes against agile principles in that it chooses processes and tools over people, who can't enjoy the benefits of agile methods, nonetheless have to invest the extra effort they demand. And almost all of it is completely in vain, as the software won't be any more of value than it would have been with traditional methods, because it will turn out to be exactly the thing that was required/designed/estimated/agreed-upon before any line of code had been written and/or tested.

Cont.'d - Quality: another agile buzzword?

2010-04-08T01:42:00.000-07:00

In my previous post on software quality, I talked about how quality has an indirect, nevertheless beneficial effect on costs and time.
Now I want to continue with the next most important factor of a project, that is the most important from the customer's perspective.
(Functionality) How improved quality helps to deliver better functionality? Interestingly enough, asking that question is already the first part of answering it. Really, there is no magic answer here: it's plain to see, that once you concentrate on quality from day one you will deliver better functionality. Your testers can focus on usabilty instead of checking and tracking trivial bugs and scenarios. Instead of the i-click-here-and-it-crahses type of bugs, you will get more of the it's difficult-to-find-that-menu-item or this-messagebox-makes-no-sense kind of bugs. Your customer also can effectively participate in the developing process, once he's not stuck trying to only install and start your program. In short it's similar to the Maslow for Programmers, but on the project's scale. If customers, product managers, developers and testers don't have to spend precious time on finding, documenting and fixing trivial bugs, they all of a sudden have time for something else, and unless they hooked up on porn beyond the avarage amount they can spend it on creating software of real value.

(Developer Team)And last but not least, I'd like to mention DeMarco's take on quality. He advocates that quality is more important for the dev team, than anyone else involved in the project. It's for the programmers' self-esteem. If you deliver quality software as a rule, you can be proud of every day's work. If you are proud, you can and want to take credit, if you take credit, you will be even more motivated to deliver quality. It's a positive feedback loop, but according to Wikipedia, it's not a bad thing:
"Positive feedback amplifies possibilities of divergences; it is the condition to change, evolution, growth; it gives the system the ability to access new points of equilibrium." It also helps team culture, and creates an atmosphere where the constant urge to improve is a common and selfexplaining thing.

Quality: another agile buzzword?

2010-04-04T05:06:00.000-07:00

Agile literature is full of explanations and techniques as of how to improve software quality. There is lot less talk about why quality is important in the first place. Even K. Beck goes only so far that quality improves trust, and that with trust everything is more simple.
I think an important issue is rarely or never addressed. The fact that users and customers don't seem to care about quality all that much.

As DeMarco puts it, if you present to your client that the software right now has a Mean Time Between Failures of 1.2 hours, and with three extra weeks of work you could reach an MTBF as high as 2000 hours, then after some hemming and hawing "the users will explain that they are as quality-conscious as the next fellow, but three weeks is real money." [1]
This is another classic example of why software industry has way less in common with more traditional industries(cars, architecture, etc) than we've been told. What is quality in other areas, if not the longevity of things? One considers a car or a watch of good quality if it lasts long without the need of repair. Well, in that sense software is always perfect, you can use it as much as you want its parts won't break or fray. Of course there is a minimum quality required, but only to the extent that poor quality doesn't render the software utterly useless, and anything above that is a bonus, that clients aren't willing to pay for.

So why are agile teams so hung up on quality then? - Well, you should ask them, but my guess is that they don't know. [if you are in agile development take a minute here and try to answer that]

I get back to that in a minute, but first I'd like to point out, that when selling your agile methodolgy to your clients, quality shouldn't be high on your list of arguments, or not at all. Quality 'per se' has little to no value to your clients. What you should try instead is to explain to them, how quality will be beneficial for the factors that they do care about, i.e. costs and time and functionality.
(Costs and time) The cost of a software in general is made up of two - not all that independent - parts: development and maintenance. It is easy to see that improved quality brings the cost of maintenance to a minimum. But what's often overlooked, that quality doesn't increase the cost of development in turn. In fact, many times it lowers that too. How is this possible? You just need to recall all the times when you were happily coding away and away, until all of sudden you hit a wall, a wall made of all sorts of different "materials": badly designed, badly laid out code; bugs hiding other bugs that have been lurking around for months only to surface at the worst time possible; performance issues that can only be resolved by completely restructuring your code which - now that you think about it - doesn't really have a structure; endless email/bugtracking ping-pong, between you and the test department; so on and so forth. That is the moment, when development cost and time estimates go out the window, and unfortunately this moment comes almost always nearing the end of the project when it is no more possible to rethink scope, deadline or effort.
I'm not saying that quality is a silver bullet, and that you will never hit a wall with agile, but I can say with confidence, that
a) chances of hitting a wall are much lower
b) even when you hit a wall, it's less dense (is made up of fewer things)
c) and the end of the project is typically not near at all
I'm saying that if you constantly strive for quality the inevitable walls of software development will become more like bumps. And that's your selling argument right there.

[Next time I'll talk about the third factor (functionality), as well as why quality is important for the developer team.]

[1] DeMarco - Lister: Peopleware

Maslow for Programmers

2010-03-12T08:33:00.000-08:00

What would the Maslow-pyramid look like if we applied it to programmers?
(Physiological Need) The starting layer is the fundamental knowledge to implement something in a given environment with given tools. This doesn't need further explanation, without this the rest has no meaning.

(Safety) The basic security of your program; that is a relatively bug-free level, where you can think of, and prepare your software for many of the exceptional situations that could arise.

(Love, Belonging) The awareness of the user and the explicit intention of creating software that has value for its users. This level also includes the intention of writing code that others can understand, designing software with aproved and well-spread concepts in mind, following industry standards and guidelines.

(Esteem) When you don't only care about clean code and design, but also about the process by which you achieve those things. Reflection is a great part of this. Through honest and regular reflection you regularly arrive to the conclusion that you need to learn more, and so you do just that. Without continuous improvement, you can't reach the fifth level

(Self-actualization) Where you can make your own contributions to the software world let they be books, theories or techniques. They could also be development tools, SDK's that aid others' work or have beneficial influence on the way others implement things.

Are you still with me? If you are, you could say, "It's all very nice, but is there a point to all this?" There might.

You have to know your tools and the programming language you use before you can look any further. This sounds trivial, but it's actually quite difficult to achieve. The problem is that many of today's development tools are so sophisticated, sport so many features, offer so many options and solutions in syntax and in libraries, that you just can't learn it from a book. You have to work on many projects, preferably in different problem domains, so that you can come across all particular features of your tool that solve your particular problems perfectly. This takes time, and luck is also a factor here. The good news is that once you have acquired that knowledge, it's only up to you to move up in the pyramid:

* You can start to think about the value of the software you develop.
* You can start looking up literature on how to design, create and test software. It's all out there.
* You can start reflecting on yourself(on your team), you can start improving your process, you can strive to become better at what you do.

Okay, the top level may be out of reach for most us. Not everyone can think as originally as Kent Beck, or write about software as systematically and meticuluosly as Frederick Brooks. But who knows? Maybe, one day you will write the next, all new version of Rhino.Mocks for all the .Net-developers to awe.

Assert First Development

2010-03-03T12:40:00.000-08:00

A good programmer writes about 10 lines of code a day. Maybe less maybe more, though it's a very interresting topic leading to names like Barry Boehm and Tom DeMarco, it's not the subject of this post. I just wanted to point out that eventhough undoubtedly useful, IntelliSense, or any autocompletion tool for that matter, is way overrated. A programmer might type in that single line of code of the hour in five minutes, or in eight without IntelliSense. I'm glad we've got that out of the way. Next, if you are a TDDer and don't already use the Arrange/Act/Assert concept, start today and you will never look back. Anyways.

"Assert First Development"[1] is about writing the assert of your test first. This gives you the means to write the act-part in a way your assert needs it, and then the arrange-part as your act and assert needs it. This way you have to think about only one thing at a time. If you do it the other way around - that is from arrange to assert - you always have to think about all three parts simultaneously. You could of course say "I'm very smart and I can think about many things at once", and I believe you can, but you could use all your cleverness to solve one problem at a time, potentially solving things even better. If not, you still haven't lost anything, you just did your job sequentially. But chances are that your assert will be right the first time around, not mentioning your arrange in which you set really only those things up your act needs, and not the things you think your act will need. Only thing you lose is some parts of your IntelliSense, but we already established that it's really not that much of a loss.
One last thought. Assert First Development is another great example of the pull-system: there is never waste produced, we don't setup mocks or fakes or anything we might not need eventually.

References:
[1] Kent Beck: Test Driven Development: By Example

Problems With And Without Honesty

2010-01-25T11:06:00.000-08:00

Without honesty there are a tons of problems. It's easy to see why Beck (not again?!) advocates openness and transparency in software development: in short, if we talk about issues honestly we stand a much better cance to resolve them. It's easy to see, but very difficult to realise.
Difficult because of the only problem with honesty: our self-esteem.

More specifically the From and the To part of honesty. The difficulty with the From part is manners and custom. We have a hard time telling our colleague he screwed up badly or has not the first clue what he's talking about, because that's what we are all used to: the not-telling. And because we could be and many times have been at the To-end of honesty and we know first hand how it feels when our mistakes are pointed out. From or To: our self-esteem gets in the way.
In retrospectives mature teams have the strength to be honest about mistakes, as I mentioned before. But it's a lot easier to tell honestly and to be told honestly about problems, when you are team. This is not to underestimate the power of collective responsibility, actually this is to encourage teams to try. You will be susprised how much more honesty you can handle as a team.
However. A team's straightforwardness is not invincible either. At the end of the day the team is made up of individuals, and their willingness and inclanation for honesty ultimately will draw the limits of the team's honesty. Team-members will have to intentionally put aside their self-esteem and pride so that the team can go the distance with honesty, can find real issues, real solutions and once again the team can improve.

Your Choice

2010-01-23T04:18:00.000-08:00

Here's the deal. The last post wasn't much of anything. I know. The reason was that there are a couple of things I want to write about, but couldn't decide which one to pick, so I wrote about something I didn't really want to. Anyways, now is your chance to decide what the next post should be about. Vote on the right, comment below, if and when.

"Perfect is a verb, not an adjective" [1]

2010-01-19T11:31:00.000-08:00

Learning new stuff is paramount in our line of work. I know, I know Cobol coders might tell you otherwise, and yeah, cobol coders might be home free, but the rest of us mortals need to keep up with the pace technology dictates. Or at least not to fall behind irreversibly much. Because then you might become the subject of Murphy's law, saying "If a hammer is all you have, everything you see will appear to look like a nail". It seems trivial, yet many software companies' don't seem to be sporting any learning or improving. Employers and employees alike seem to be confident in and/or content with their current experience and skills. Obviously it's extra effort short term to spend time and energy to learn new things, but with newer and better tools and techniques you can deliver better sotfware in shorter time. In fact the benefits and the profit are so plain to see, that it doesn't need any further explanation or examples, but then again this post would be ridiculously short.

Once you start looking up new stuff, or rather - because it might be something old - stuff you don't know, it becomes second nature, and you don't even notice all the advantages it gives you, or the way programming becomes easier with recently acquired knowledge. So it's difficult to pick any one example from our past year but I settled for mocking frameworks.
When we got to know TDD we dived into writing lots and lots of incomprehensible unit and functional tests nobody understood or was able to maintain. Then we stumbled upon mocking frameworks, and how we could live without one, beats the hell out of me now. We started to use Rhino Mocks, but soon enough our unit tests once again started to look like a can of worms, so this time intentionally searching for a cure, we found out about the AAA-concept (it's short for arrange/act/assert, but I can't find the exact link anymore; go google it up!). Finally we learned that we can use the lambda-expression to make our code even cleaner and readable. We learned and learned till unit testing and object mocking is less and less of a technical problem, and we can concentrate more on what we are trying to express and accomplish with our tests.
Again this example it's not even about new stuff, it's just stuff we hadn't known before. Keep learning and remember there are other useful tools than just the hammer.

References:
[1] Kent Beck: Extreme Programming Explained 2nd Edition

Transformers

2009-12-29T10:27:00.000-08:00

Much of the agile literaure is focused on how difficult the transition from traditional to agile could be. They address the problem from many angles describing techniques how to help programmers to make peace with pair programming, how to get better at incremental design, some even suggest to use Organizational Patterns to introduce some of the practices to a team or a company. They all seem to be empathic with people who as a rule fear and loath change.
I don't see it that way. I actually think being a true "waterfall guy" helps you to accept agile, and helps you to execute XP practices and to a better outcome too(from this point on I'll be referring to XP, but probably most of what I have to say applies to other agile methodologies as well)

Having had been a true waterfall guy myself, with my precious UML diagrams, my very own modules and an ever strengthening habit of working alone on a laptop in pubs and cafes only occasionally checking in my work for my team to see, I had no problem at all transitioning to XP -- quite the contrary.
I enjoyed Pair Programming from Day 1, because I realised how much easier it is to pick one from the many possible ways to solve a problem, if you have someone competent to discuss it with. I learned not to fear to be stuck anymore, simply because it happens almost never, and even if it does, it's far less frustrating when you have someone next to you to share the pain. Becuse of all the implicit and explicit problems I had with the tradtional way, I appreciated Pair Programming a lot more then I would have otherwise.
To be able to do Incremental Design it's good to have had done a few BDUF yourself. It helps you to see the big picture - which some false interpretation of incremental desgin denies to be allowed - and it helps you to come up with sophisticated solutions quickly. But most importantly once you have seen first hand what's wrong with BDUF, you are ready to like Incremental Design for all of its benefits.
TDD is again something you appriciate more coming from a Waterfall world, because then you know how fustrating the bugfixing phase normally is, fixing and refixing bugs for months. You also know how frustraing and shameful it is not to make changes to code you hate and not to refactor at all due to either fear of breaking something, or an explicit order from your bosses, with something about changes are risky at this stage of the project, but the right stage for a change never seems to come. In which BTW they are absolutely right: without an extensive test harness, you can never be sure, even a little, that the changes you're about to make won't break the whole system.
All in all, the skills you learned and the bad experiences you had with waterfall all will make you a better agile developer.

Oil And Water, Or Are They?

2009-12-22T05:27:00.000-08:00

Tester and developer couldn't be further apart, both of them fighting all the time without knowing any better. As one of our former tester put it, we have different goals: developers want the software to work, testers want it to break. For a long and regrettable time we shared this belief. But then we thought with agile it is over: bringing the tester closer to the team, both locationally and spiritually, would bring everyone to the same goal, i.e. to create quality software. It didn't work out as expected. Maybe the shared goal still wasn't shared enough, but I think we were missing a more important ingredient.
Fortunately we had learned our lesson, and with our new tester we tried harder. Once again Kent Beck helped us to find the missing pieces: Respect and Trust.

Though in his lecture he talks specifically about how trusty realtionship between developers and clients can help both parties to concentrate on creating values, I watched the inner workings of respect and trust in a tester-developer relationship.
In a timespan of a couple of 2-week iterations, our tester became a full-member of our team, thank to trust and respect. Quite rapidly we have arrived to the point where she trusts the developers enough to dare to ask them about the observed and expected behaviour of the system. She knows now she can be sure that a bug-candidate won't be dismissed due to programmers' arrogance or ego. She also knows she can freely ask all sorts of questions and won't be shushed or ignored or made fun of.
On the other hand we developers learned to respect her and her view on our software, and we learned not to discount whatever she has to say based solely on the fact that she didn't write the code, and in fact wouldn't even know how to.
Few minor, though important things at last. This productive relationship couldn't spring to life without our seriousness toward bugs: in an attempt to avoid the mini-waterfall effect we try to act quickly upon new bugs, this is not only practical from the technical point of view but also expresses respect, and thus strengthen trust. Unit and functional tests makes it possible that very few already working features break due to a fix of a new bug, again strengthening trust. Finally, our relatively fast build closes a very short feedback-loop of finding, fixing and retesting a bug creating a productive and positive atmosphere.

The True Weight Of An Iteration Board

2009-11-23T13:09:00.000-08:00

An iteration board carries lot more than its actual weight. It is there every day to remind you of all your mistakes and weaknesses. Sure, many people envy us for our friendly atmosphere and the unusually humane working environment we work in. What most people fail to see is our constant struggle to do our best. They don't see the cruel honesty that forces us to face our shortcomings day after day. There has not been an iteration, however successful, going by at the end of which we didn't identify issues in our process, approach or skills that we needed to resolve.
This honesty could be a real burden, especially for one with a programmer's ego, if you don't reach out to Kent's Principle of Opportunity, that states that we need to look at problems as opportunities. In this case as opportunities to improve.
Also don't forget, the problem is not the iteration board and its bug-cards for instance. Iteration boards and burndown-charts can solely be indicators of something going wrong, they are not the problem, nor the cause. So next time when you look at a burn-down chart that is not steap enough, or an iteration board that is not "green" enough, don't rush to find solutions to make it steaper, or don't try to organize your work, so that you can score more green points. Try to find the underlying problems, the more the merrier, forget your ego, and think of all the improvements you now have the opportunity to make.

The Navigator

2009-11-07T13:11:00.000-08:00

Pair programming is one of the key practices of XP, and so there is a lot to know and learn about it(which you can start by reading Pair Programming Illuminated by Laurie Williams, Robert Kessler). Let's focus on the navigator for now.
The navigator's role in my view is to look ahead: while the driver is to keep us on the road without any accidents, the navigator is to know where we're going and why, also what we're gonna do when we get there.
I figured that the best way I can accomplish this as a navigator is if I sit back and don't even watch what my driver is typing.

Don't get me wrong, it's not without merit when the navigator and driver don't talk much, just simply understand each other through bits of code and a few words, though this benefits the team more on the personal and social levels, helps to build mutual respect, team spirit and so on.
But the navigator's sort of turning away from the screen makes it imperative for him and the driver to start talking about the problems and solutions. A lot of good things can come out of saying something out loud. For instance you can realise you can't say it: it's too blurry to be said, and then chances are that your solution or your model of the problem is way too complicated. Other times when you try to explain your implementation out loud, you realise that it's just simply wrong.
In our praticular office layout there is another, quite relevant side effect of my turning somewhat away from the monitor: I get to see other parts of the room: the iteration board, the release board, the burn-down chart, our product manager, our tester, our build machine - all of which remind me to look at the problem at hand from many, different perspectives: all of which I might have forgotten about, had I watched my driver and his code unneccessarily closely...

So you can try it next time when you're the navigator and let me know how it turned out...

Case closed

2009-10-23T05:06:00.000-07:00

Many heated arguments about Test-Driven Development have certain recurring reasoning on both sides. In general, developers have a hard time accepting that they need to think differently about implementation and design and/or have to alter exisitng design in order to be able to write tests. "I would need to introduce this indirection only for testing" or "There is no need for this kind of hierarchy to implement the features in the production system", and their alikes are common critiques of TDD.
Inexperienced TDD people almost invariably try to tackle these with answers like "But your design will be better this way" or "Modularity is your friend" or "Indirection is a way to build flexibilty into your code".

Obviously all of these are correct and nothing new. SOLID for example is a well-known and universal set of principles supporting the answers above. However most of the defenders of TDD are missing the point.
The point is this:
Once we've agreed that unit testing is a very efficient way to create quality software and we want to do it, there is no room for further discussion. You just have to think about the testability of your code as any other requirement you have to implement. Don't worry, you will get all the earlier mentioned: flexibility, maintainability, comprehensibility, and so on. But that shouldn't have influence on this praticular matter, because you are required to write testable code anyway. There is really nothing else to it.
Once you can get your head around this, your life as an agile programmer will be much, much easier. Mine already is.

Traps Pt.1.: Rope-A-Dope

2009-10-17T12:55:00.000-07:00

Many times in software development you have a simple problem, which has an easy solution. Or so it seems. You start solving the problem, but it turns out first you have to solve a different problem. Not a difficult one either, so you start working on it, but it's not long before you realise that before you could solve the second one you have to start with a third, all new problem. Again, not a very complicated one. Before you know it it's six o'clock in the evening, and you'd forgot all about the original problem, let alone solving it. Let's call it the Rope-A-Dope Trap. It's difficult exactly becuse it seems easy all the way. Every time just one small step away. There was just no point where you could have said no.
A possible escape from this trap is applying the Rule Of Three(or Two).

There are always obsticles creeping in when solving software problems, it's inevitable. But whenever you find yourself face to face with the third "creeper" in a row, you need to abort what you're doing, take a step back, a deep breath and a very good look at your original problem in light of the three unforseen ones. Rethink, regroup and start over. Starting over could mean a different approach, design, tool or even different requirements.
So watch out next time for a Rope-A-Dope. And don't forget that it could occur on many levels too. A method, a feature or a whole solution.