GIANT ROBOTS SMASHING INTO OTHER GIANT ROBOTS

Shhh! Your Test::Unit backtraces are too noisy!

Posted by Dan Croak

Dec 03

For more details on the history of this little open source effort, see James Golick’s post at James on Software.

I’m a child of the Information Age, over-stimulated with a tiny attention spa… hey, a blue car!

Therefore, I love Rails and Agile processes for focusing my mind so I don’t get distrac… haha, those squirrels are chasing each other! I’ll call the little one Bitey.

However, Test::Unit backtraces are too noisy for my tastes. Particularly after seeing the Rubinius backtraces at RubyConf a few weeks ago, I’m about as interested in noisy backtraces as I am in returning to developing in C#.

90% of this typical backtrace will not help you hunt and kill bad code:

  1) Failure:
  test: logged in on get to index should only show projects for the user's account. (ProjectsControllerTest)
  [/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/assertions.rb:48:in `assert_block'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/assertions.rb:500:in `_wrap_assertion'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/assertions.rb:46:in `assert_block'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/assertions.rb:63:in `assert'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/assertions.rb:495:in `_wrap_assertion'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/assertions.rb:61:in `assert'
  test/functional/projects_controller_test.rb:31:in `__bind_1196527660_342195'
  /Users/james/Documents/railsApps/projects/vendor/plugins/shoulda/lib/shoulda/context.rb:98:in `call'
  /Users/james/Documents/railsApps/projects/vendor/plugins/shoulda/lib/shoulda/context.rb:98:in `test: logged in on get to index should only show projects for the user's account. '
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/testcase.rb:78:in `__send__'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/testcase.rb:78:in `run'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/testsuite.rb:34:in `run'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/testsuite.rb:33:in `each'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/testsuite.rb:33:in `run'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/testsuite.rb:34:in `run'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/testsuite.rb:33:in `each'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/testsuite.rb:33:in `run'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/ui/testrunnermediator.rb:46:in `run_suite'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/ui/console/testrunner.rb:67:in `start_mediator'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/ui/console/testrunner.rb:41:in `start'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/ui/testrunnerutilities.rb:29:in `run'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/autorunner.rb:216:in `run'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit/autorunner.rb:12:in `run'
  /System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/lib/ruby/1.8/test/unit.rb:278
  test/functional/projects_controller_test.rb:36]:
  one or more projects shown does not belong to the current user's account.
  false is not true.

Noisy backtraces must be ruthlessly silenced like political dissidents in Stalinist Russia. This much is clear.

So on Tuesday night, I delivered my dictatorial analogy to some good folk in the Boston.rb group and received an overwhelming response: “Croaky, ease up on Stalin references, but this is a good idea.” So, at the inaugural Boston.rb hackfest, we scratched our itch, with especially good direction from Joe Ferris. As the week progressed, James Golick and I polished the resulting code into a gem which I am pleased to announce… now.

Quiet Backtrace

Install the gem:

sudo gem install quietbacktrace

Run your Test::Unit tests:

  1) Failure:
  test: logged in on get to index should only show projects for the user's account. (ProjectsControllerTest)
  [test/functional/projects_controller_test.rb:31
  vendor/plugins/shoulda/lib/shoulda/context.rb:98
  vendor/plugins/shoulda/lib/shoulda/context.rb:98
  test/functional/projects_controller_test.rb:36]:
  one or more projects shown does not belong to the current user's account.
  false is not true.

Ooh la la! Now we’re cooking with gas. However, those shoulda-related lines are cluttering an otherwise perfect backtrace. Luckily, Quiet Backtrace is designed to be extended by calling two types of blocks that yield one line of the backtrace at a time.

Silencers and filters

1. Silencers let you specify conditions that, if true, will remove the line from the backtrace.
2. Filters let you use Ruby’s succulent set of String methods to modify a line by slicing and stripping and chomping away at anything you deem ugly and unnecessary. (such as the :in `__bind_1196527660_342195’ in the original example)

Say you want to remove Shoulda-related lines… you create a new silencer and add it the Array of backtrace_silencers:

1
2
3
4
5
6
7

class Test::Unit::TestCase
  self.new_backtrace_silencer :shoulda do |line| 
    line.include? 'vendor/plugins/shoulda'
  end
  
  self.backtrace_silencers << :shoulda
end

Re-run your tests and bask in the sweet sounds of silence:

  1) Failure:
  test: logged in on get to index should only show projects for the user's account. (ProjectsControllerTest)
  [test/functional/projects_controller_test.rb:31
  test/functional/projects_controller_test.rb:36]:
  one or more projects shown does not belong to the current user's account.
  false is not true.

Exquisitely sparse. Quiet Backtrace clears distractions from the “getting to green” TDD process like a Buddhist monk keeping his mind clear during meditation.

Getting noisy again

On occasion, you’ll want to see the noisy backtrace. Easy:

1
2
3

class Test::Unit::TestCase
  self.quiet_backtrace = false
end

You can set Test::Unit::TestCase.quiet_backtrace to true or false at any level in your Test::Unit code. Stick it in your test_helper.rb file or get noisy in an individual file or test. More flex than a rubber band.

Using Quiet Backtrace with Rails

After you have installed the gem:

cd rails-app-folder/vendor/gems
gem unpack quietbacktrace

require 'quietbacktrace'

Quiet Backtrace will now work with your tests, but because this gem is meant to work on any Ruby project with Test::Unit, it does not turn any Rails-specific silencers or filters on by default. However, there is one of each, ready to be switched on, that remove the most dastardly lines.

Add these lines to your /test/test_helper.rb file to get perfectly clean Rails backtraces:

1
2
3
4

class Test::Unit::TestCase
  self.backtrace_silencers << :rails_vendor
  self.backtrace_filters   << :rails_root
end

Ongoing development

The most up-to-date information will always be at the Quiet Backtrace project page. Bug reports and patches are welcome at RubyForge. Talk about it on the mailing list. It’s is a safe place. A place where we can feel free sharing our feelings. A nest in a tree of trust and understanding.

On a related note, the Boston.rb hackfest (part deux) will be held on December 11 at thoughtbot’s office. Any Rubyists in town that night are welcome.