not worth it

Posted by jcarroll

Jan 06

Behaviour-Driven Development.

The latest fad thats been around for a while but seems to be picking up a lot more lately. I messed with it and blew it off a while ago but that was a while ago, so I decided to check it out again and see what all this malarkey is about.

From behavior-driven.org

It is not too surprising that it takes apprentice TDD practitioners a while to realize that TDD is not about testing when all of the nomenclature surrounding it is described in terms of testing. The aim of BehaviourDrivenDevelopment (BDD) is to address this shortcoming. By using terminology focused on the behavioural aspects of the system rather than testing, BDD attempts to help direct developers towards a focus on the real value to be found in TDD at its most successful.

So BDD was created because it was hard to teach people that they had to write tests for code that didn’t exist yet? So all BDD does is change the vocabulary to make it seem like you’re not writing tests but “behavioural aspects” of your system?

I get how when practicing TDD your test is the first client of the code you’re about to write; essentially developing interfaces the way you’d like to use them, so it is design.

From BDD advocate Dave Astels

Does that mean you write tests? No. It means you write specifications of what your code will have to do. It means you specify the behaviour of your code ahead of time.

Ok that’s TDD to me. Tests are a way of specifying what your code will do. But now I have to refer to them as “specifications”?

Let’s look at some code now.

Here’s the popular Ruby BDD library rspec. The following code was taken off their site.


describe Account, " when first created" do

  before do
    @account = Account.new
  end

  it "should have a balance of $0" do
    @account.balance.should eql(Money.new(0, :dollars))
  end

  after do
    @account = nil
  end

end

Ok #describe is basically defining the equivalent of a TestCase in xUnit. Now #before and #after are equivalent to the #setup and #teardown methods in xUnit. And #it is the equivalent of a test method in a TestCase subclass in xUnit.

I have 2 problems with this

I have to learn and start using this new language to talk about tests
I have to learn a whole new syntax to write tests

I’ve used xUnit in a number of languages, its what I know. I like subclassing TestCase, getting #setup and #teardown for free and knowing that if I name my test methods starting with “test” that they’re going to get executed automatically. That is simple, obvious and what the majority of developers out there know.

I jumped on Rails because it took building web applications clearly to a new level, but to me BDD is not doing the same when it comes to testing. Somebody give me a reason to use it.

Thoughtbot

thoughtbot is a technology consulting firm that provides web application development and design services. We focus on building modern systems, embracing good ideas and delivering elegant solutions.

Interested in learning Rails?

Comments on this post

Jan 07

Ryan Allen said,

I use RSpec to write my ‘tests’ because I prefer it’s API and it’s excellent SpecDoc output. I still describe them as tests and not specs.

For those who have carefully read Test Driven Development By Example written by the venerable Kent Beck, I think they’ll find that he does in fact describe that testing behaviour is the key to the practice. I believe the problems that BDD are trying to solve are probably habits developed (and taught) by people who aren’t thinking a whole lot about why they’re actually testing. Kent’s a deep thinker and it is reflected in his writing.

So yeah, while I agree with you that BDD is just another way of describing TDD (and perhaps thus stealing the lime-light? who knows), RSpec is a nice API. But for what it’s worth I still refer back to Kent Beck’s writings when I want to deepen my knowledge and understanding, and I find I get new insights each time I do.

Finn Higgins said,

Here’s my take: documentation.

spec:doc produces a very concise and readable specifications document if you’re even mildly smart about the way you phrase your spec descriptions. So if you want to develop user documentation then a full set of RSpec tests in your source tree gives you a versionable, diff-able set of specs that can be given directly to your documentation people.

Want somebody to update the user manual from v1.0 to v1.5? Just pull the two versions of the source tree out of version control, run spec:doc on each, diff the results and give your list of changes to the technical writers.

For me that’s enough of a win.

sandofsky said,

This is the same argument as to why we use the qwerty keyboard when dvorak is technically better.

If I knew someone learning TDD, I’d direct them to rSpec. At yellowpages.com, our tests are test::unit because that’s how they began, and we’re writing them behaviorally anyway.

The big technical advantage of rSpec is its output. You can print spec output to help communicate with a client. You can write “pending” specs as a sort of todo list.

Plus some metaprogramming magic encourages beautiful code. For example, it adds a matcher for methods ending in a question mark, so

1234	assert object.valid?

becomes

1234	object.should be_valid

The real “not worth it” i’m wonder about is 1:1 coverage.

Damien said,

It seems most people here use it for doc generation. Ok it may be cool, but don’t you right Specification before writing your tests? Then your specification should be the base of your documentation, isn’t it?

Well that’s how we work at Dexem, as we already have specifications written, we don’t need to re-write it after coding, we just iterate during development and make sure everything is synchronized.

Ben Mabey said,

“So all BDD does is change the vocabulary…”? Yes. That is the point. I can understand how such a seemingly small change would merit learning a new API when you already have a lot of time invested in xUnit but from personal experience I can tell you that it is well worth it. The language that you use changes how you think about the system. This is much like how when modeling a business domain you should strive for a ubiquitous language (Domain Driven Design by Eric Evans) that domain/business experts and developers can communicate in but will also permutate down into the code. I think that the vocabulary of BDD is much closer to the domain of what we, as developers, are really trying to accomplish when we are specifying what a system should do. That said you do not need to learn a new framework to do BDD. Proponents of BDD have always held that their goals and methodologies were nothing new, but rather BDD is TDD at its best. As sandofsky pointed out, they were using test:unit to test behaviorally. So any xUnit framework with a nice mocking framework is really all you need. The sad thing is that most shops are not doing TDD, let alone doing it right. BDD is, in a way, an effort to crystalize the best practices of TDD and make it more approachable.

RSpec helps a great deal in keeping me doing BDD and really testing behavior. The DSL of rspec is really nice and can be picked up quite fast so their really isn’t much of an investment needed. Aside from the DSL rspec, as mentioned above, has lots of added bonuses. RSpec’s new story runner, for example, is very exciting. As an Agile/XP shop this opens a whole world of possibilities in making the feedback loop between you and the customer even tighter.

Morgan said,

Greetings, Pure BDD is for people who never have to touch the project again after the initial coding push. I’ve never seen a ‘spec’ced project where there weren’t huge swaths of functionality that weren’t tested, because it didn’t fit the simplified concepts of ‘behavior’ or ‘spec’. In the Rails world, views, helpers, and routes are common victims of this blindness, as are database-tuned methods (because you can’t ‘spec’ performance).

The wording change is also meant to appeal to people who don’t like the idea of writing ‘tests’, want something sexier on their resume, or think that testing is only for ‘testers’.

The ‘self documenting’ part is bunk. Maybe for projects consisting entirely of new code, but as soon as a project is complex enough, or involves numerous other engineers, it becomes a ridiculous conceit to think the tests are a specification. Often they don’t even specify what’s most important (and causes the most bugs!) in the majority of programs: actual user interaction.

You’d be better off mentally if you recognize that you are writing TESTS, whether for functionality or bug regression, and you should be documenting design in actual documentation, preferably as close to the actual implementation as possible, so you can hope that when the design changes (even if the behavior doesn’t!), the documentation has a better chance of reflecting it.

Accept that you write tests, and that writing tests for your code is a good, decent, and important thing to do, whether it’s before or after you write the code. Having tests will make your life easier eventually, but don’t make it something it’s not.

It’s great (and feels good!) to write tests first when you’re initially building software, but for goodness sakes, don’t turn it into a religion with rituals and incantations.

‘spec’-based testing brings nothing to the table, and only confuses the issue by pretending to be yet another silver bullet.

— Morgan

Blake said,

I find there are several advantages of RSpec over Test::Unit: * Writing specs comes much naturally—the control value coming first in an assertion feels totally unnatural. I suspect that this is generally true for most developers, as I’ve seen just about every new test author write assertions in reverse order. * RSpec provides much cleaner separation of concerns and allows you to deal with your view, helper, and controller logic separately and with independent specifications. * RSpec embraces mocking and stubbing at a very deep level which enables you to write much more focused examples that run at extremely high speeds. There is a definite tipping point for Rails codebases where your test suite becomes a productivity barrier. Mocking and stubbing to remove processing and database access from other layers not currently under test is the only way to maintain acceptable speeds. * RSpec enables a higher level of abstraction through the use of custom matchers and you essentially get an ever growing library of matchers for free from your methods that end in question mark. i.e. user.admin? => user.should be_admin, user.should_not be_admin. * You can author integration tests in plain English with the new Story Runner. This makes it much, much easier to step into the user’s shoes and think about system behavior expectations and program logic without having a head full of code and implementation details. It is liberating. * But in what is perhaps the biggest advantage, BDD changes the central question from “does this thing work?” to “is this designed properly?”. Proper execution is a given, the testing is implicit in the design activities. Designing proper abstractions and building clean, robust API’s is much harder than making something work. So let’s focus on the hard problems and let the secondary ones fall out.

You of course can pull of this off with Test::Unit and if you already are, then there’s no need to switch. But this isn’t the golden path laid out for you by the framework, its a set of learned conventions and architecture you have likely built up in your own application. RSpec, like Rails, brings together many threads into an opinionated, tightly integrated package.

Morgan – Your analysis of RSpec and BDD is both ill-informed and unnecessarily inflammatory. First, the larger and more complicated a project is the more valuable a test/spec suite becomes for documentation. Its the quickest way to find out how things work, coding conventions in use, who owns what components (i.e. from svn blame), and generally how the system is designed. Secondly, RSpec actually provides tools for specing/testing Rails components that Rails itself does not ship with. There is no view or helper test case in Rails. There are view and helper behaviour types in RSpec. Furthermore, it embraces a separation of concerns that ensures you are focused only on one piece of code at a time, which is polar opposite of the Rails “test everything that happens in a request here” functional testing modus operandi. Furthermore, Specing brings quite a bit of value to the table. First, you can actually show well written spec code to a project sponsor. It has full sentences and reads like English. With the Story Runner in RSpec 1.1, you can actually write full integration stories in English that become executable specifications of behavior that guarantee the system is operating correctly. Where’s the equivalent of that in the Test::Unit world? No one has claimed BDD as a silver bullet. Only as an evolution of the lessons learned from 15+ years of TDD. Like all technologies, RSpec and BDD build on what has come before to address shortcomings and elevate the craft. I suggest you educate yourself a bit more on the technology before issuing loud public condemnations in the future.

Cheers, Blake

Tammer Saleh said,

Jared -

I agree completely. It seems to me that BDD is just a new set of terminology, but that it offers nothing that good TDD doesn’t offer. The fact that BDD is often described in vague terms solidifies that opinion for me.

I understand that there are a lot of coders out there doing poor TDD, but that doesn’t warrant a new vocabulary – rather just further education.

Jon Yurek said,

If your only documentation is what you’re generating from your specs, you’ve got terrible documentation. While it may be better than no docs, there is no substitute for proper communication.

I think Story Runner is a horrible idea: English is not a programming language, and tests and specs are still programming.

James said,

@Jon Yurek: re: story runner. If your customers read and write ruby I would agree with you. All of mine seem to prefer English though. What did your customers not like about it?

bryanl said,

It always seems to come back to the same thing, “I don’t get it, so it must not be any good.” With all new and different ideas come controversy which is a good thing, because it forces people to provide critical evaluation which we all benefit from.

BDD is nothing new. Lots of people have been doing it for a while without really knowing they were doing it. Or maybe they knew and they didn’t choose to call it BDD. There isn’t even enough information on the behavioral development website to demonstrate why you should or shouldn’t be using it.

Rspec is a different syntax. I don’t buy into it being hard to learn as it was designed to be easy to pickup. In reality, writing specs is what I have been doing in xUnit for years before I had even heard of Rspec. If you could write Shoulda, you should be able to at least understand Rspec.

As to what Jon Yurek posted:

I agree that English is not a programming language in the traditional sense, but you can use English to create a list of actions that everyone can understand, and this is why I like the stories. They are the easily understanable goals of my website or application. I use them to communicate with the owner of the application and tighten that feeback loop. My goal as a lazy developer is communicate my actions as simply as possible so that they are understood. I’m not into black magic development, and the stories allow me to consistently demonstrate functionality in an easy to understand fashion.

I always ask developers why they are writing the code they are currently working on? If you can articulate the reason, why not put it in a spec? And next time they start writing code, you could save yourself some time by putting it a spec yourself. With stories you are just abstracting it to a higher level. You write stories to describe functionality, and your lower level specs just describe your implementation of that functionality.

PS. I’m giving a talk on this exact subject at Acts as Conference next month down in Florida.

PPS. I think I’m just gonna copy this entire rant into my blog over at http://smartic.us, so feel free to comment over here or over there. And I’m not scared of someone telling me I’m wrong either :)

Joe Grossberg said,

Maybe it’s a matter of taste, and not a matter of right/wrong/one-size-fits all.

If BDD makes you enjoy coding and results in better software, go ahead and . If BDD is a pain in your ass and doesn’t improve your work, then choose a different approach.

I think that we can all agree that testing, documentation, communication and being organized are good. Whether BDD is an ideal way to meet those needs? That’s a matter of opinion.

If BDD makes you enjoy coding and results in better software, go ahead and use it. If BDD is a pain in your ass and doesn’t improve your work, then choose a different approach.

Kerry said,

I’ve been meaning to try out a BDD framework and was looking at shoulda after reading about it in the Advanced Rails Recipes book. I understand that BDD is just TDD with different language, like the “should” in shoulda… I like that, but from what I gather Thoughbot does not really see the value in that.. will shoulda even be maintained in the future? I am just totally confused… If the new vocabulary adds nothing new why should I even bother with shoulda or any other framework with new vocabulary? Quite honestly I see learning shoulda just as much as a stumbling block as rspec- so which one should I play with so I can form my own opinion about BDD?

If your customers read and write ruby I would agree with you. All of mine seem to prefer English though. What did your customers not like about it?

Why are you customers writing your tests? You customers tell you what needs to be done and you write the tests. If your customers are programmers, then maybe I would expect they could write the English correctly enough to be transferred into a test, but they’re not or they wouldn’t be hiring you.

What I’m saying is, I’m very skeptical that I would be able to rely on the tests generated from those stories as a complete and cohesive suite without developer intervention. And if I’m intervening, the utility of it drops very quickly.

I agree that English is not a programming language in the traditional sense, but you can use English to create a list of actions that everyone can understand, and this is why I like the stories.

I completely agree. But that’s as far as the stories should get.

Justin Blake said,

It’s all pretty much been said already, but my reasons were (and are):

More readable tests and stories Beautiful spec output Ability to mark specs as pending Nested behavior blocks Ability to test helpers and views right out of the box The language/API just suits the way I think better

Granted most of that is specific to rspec/rails as opposed to just BDD in general, but, well, there you go.

Apologies for the poor formatting of my last comment, as well as the double post. Just thought I’d let you know the live preview for the comment form is not an accurate representation of what will be displayed. My list looked fine in the preview…

Brian – who said anything about not being able to understand rSpec?

Dan Croak said,

My $0.02 …

I’m fine with the term BDD and understand it to mean “good TDD”. I think we achieve that admirably at thoughtbot with Shoulda, Quiet Backtrace, and disciplined practice.

I don’t mind learning a new API as long as it fits my brain. I like the separation of concerns (testing the views, controllers, and helpers, in particular) in RSpec.

I’m not sold on Story Runner.

Just to clear up some misunderstandings… The RSpec story runner does not have to be ran with the plain text stories. You can write your code inline with the steps. In the rails context all the story runner is is a wrapper for the rails integration testing library. Prior to the story runner you would do all your unit tests of the models, controllers and view but you would then need to use test:unit to do your integration testing. When you make heavy use of mock objects, testing the interaction between the layers, as rspec advocates (although it is not necessary) then integration tests are very important because they are the only tests that will run through the entire stack. So, the original goal of the Story Runner was to bring the same vocabulary and style to integration tests that rspec had done already for unit tests. Having the story runner running integration/acceptance tests helps drive your development and it is really what BDD is all about. TDD/BDD is not just about having tests for all your code, but rather serves as a design process in and of itself by encouraging small steps. Again, this is not unique to BDD and rspec. The story runner simply helps me with this process. I’m sure you are doing something similar with shoulda for your integration tests so in the end, like some one said, it is personal preference. I just thought I should explain what the original motivation for the story runner was.

Pat Maddox has some good posts about the story runner in this context: http://evang.eli.st/blog/2007/9/1/user-stories-with-rspec-s-story-runner http://evang.eli.st/blog/2007/10/8/story-runner-top-to-bottom-screencast

Jeremy said,

I think that claiming BDD is good because it’s good documentation is horrible. Of course, this is a pretty prevalent mindset, but it is a sad case indeed. Tests/specs are useless as documentation. Why do I care how your super_secret_internal_method functions? There’s too much noise and not enough explanation.

I think Story Runner is a big fat waste of time. You spend just as much or more time setting up the backend stuff that it needs to actually run your stories that it would’ve been quicker to have just written a nice, readable spec. My customers should/would never update my specs. Coding is my job, so why am I trying to share that burden with the client?

Stories are useful but I don’t think it’s useful to try to shoehorn English into a problem domain where it doesn’t belong.

As for the value of BDD, it’s just a new suit on an old soldier. It’s a great way to teach TDD to people, but I also find it makes it easier for me to write my tests. There’s a much shorter jump from brain to code when I can think “This code snippet should give me 13 and delete a record” instead of trying to assert this or that about my code.

But I do find that I do prefer Test::Unit when I’m writing tests for existing code. This makes more sense to my brain: asserting that the code will give me such and such value. I’m testing code at this point rather than specifying behavior for unwritten code.

Of course, that’s just a personal mental perception. There’s no inherent value for either package for either task. I think it’s a both/and issue rather than either/or.

Pius said,

Ruby.

From http://www.ruby-lang.org/en/:

A dynamic, open source programming language with a focus on simplicity and productivity. It has an elegant syntax that is natural to read and easy to write.

So Ruby was created because it wasn’t fun to write clean code? So all Ruby does is change the vocabulary to make it seem like you’re not writing code but some weird sort of wannabe English to implement your system?

Java and Ruby are both Turing-complete, so it’s just not worth it to waste time learning Ruby.

I have 2 problems with it:

1. I have to learn and start using this new language to talk about code (metaprogramming, closures, etc)
2. I have to learn a whole new syntax to write code

I’ve used Java in a number of projects, its what I know. I like writing my own getters and setters, getting type safety for free, and knowing that if I create a new class, no module will re-open it. That is simple, obvious and what the majority of developers out there know.

You misunderstand. BDD (in the form of Rspec/spec:doc) is good for documentation because it provides a concise, readable spec that can be used to implement documentation and track changes to it. Not because it actually produces documentation – it does an even worse job of that than RDoc does from a user point of view.

It does, however, produce an output that (assuming reasonable test coverage) fully describes the expected behavior of your software system. A complete user manual should include documentation for all of this behavior, so while it doesn’t provide you with any usable documentation it does provide an excellent agenda for the creation of it.

I’ve done some technical writing in the past, and being given a sane spec:doc document would present an excellent starting point for blocking out a user manual – particularly when combined with the story runner, which describe typical user tasks. It’s by no means any kind of finished documentation, but give a smart writer all that information and it’ll provide an excellent brief for writing user docs.

Doubly so when you’re trying to revise a manual for a new release. How do you know what to change to cover new behavior of the latest release? You look at the changelog, of course. Where does the changelog come from? Well, you can spend ages writing it or you can diff the specs and annotate where more detail is required. That’s powerful, because it means that you aren’t duplicating effort: your expectations of your software’s behavior is all located in one place (the specs description structure) which can be used to inform both your automated testing and your user documentation. It certainly seems much more DRY than logging expected behavior separately in plain-text specs, a changelog and machine-readable tests all stored in different, unrelated locations.

Sorry, comments are closed for this article.

Come “ride the toad” on Hoptoad, the app error app.
Thunder Thimble: Brand monitoring for social media.
Widgetfinger: Simple content management for simple websites.
Tee-Bot, funny shirts your friends won't understand!
Umbrella Today: “It’s like totally the simplest weather report ever, Julie.”

GIANT ROBOTS SMASHING INTO OTHER GIANT ROBOTS

not worth it

Posted by jcarroll

Jan 06

Comments on this post

Jan 07

Ryan Allen said,

Jan 07

Finn Higgins said,

Jan 07

sandofsky said,

Jan 07

Damien said,

Jan 07

Ben Mabey said,

Jan 07

Morgan said,

Jan 07

Blake said,

Jan 07

Tammer Saleh said,

Jan 07

Jon Yurek said,

Jan 07

James said,

Jan 07

bryanl said,

Jan 07

Joe Grossberg said,

Jan 07

Joe Grossberg said,

Jan 07

Kerry said,

Jan 07

Jon Yurek said,

Jan 07

Justin Blake said,

Jan 07

Justin Blake said,

Jan 07

Tammer Saleh said,

Jan 07

Dan Croak said,

Jan 07

Ben Mabey said,

Jan 07

Jeremy said,

Jan 07

Pius said,

Jan 07

Finn Higgins said,

© 2000 - 2009 by thoughtbot, inc. written by a bushel of tiny robots

Thoughtbot

Archives

© 2000 - 2009 by thoughtbot, inc.
written by a bushel of tiny robots