Wednesday, January 12, 2011

Relishing in Perfection Part 3 -- Peeling the Raw Requirements


Installment three of "Relishing Perfection", a series on doing BDD with Cucumber, "the right way". If you are new to this series, I recommend you start with the introduction.

Writing Cucumber Features

The first step is projecting the general set of requirements into a set of User Stories. In the BDD world fueled by Cucumber, a Feature embodies the bulk of what a User Story is. This work is the essential analysis of the Business Analyst role: distilling hopes, wishes, a nebulous dreams into concrete, actionable goals. The practice of the actual analyst is a topic for another discussion; in this line of thinking, we are going to focus on the actual articulation of user stories (as Cucumber Features) and how that articulation is affected by the fact that Features are executable. The following are some guidelines and specific practices to help guide the authoring portion of this effort.
An Appropriate Nod to Dan North
Dan North, the gentleman who coined the term Behavior-Driven Development, has just posted an excellent and more rigorously minded model for looking at the very topic of choosing the right phrasing for Cucumber Features. You will do yourself right by studying his insightful blog entry Whose domain is it anyway?. The short short is to minimize the number of domains from which you take language to author your Features. Dan argues (and I see the same thing) that the more domains you add in the more concerns that are included, the more likely something is going to change and thus the more brittle your Features. I discuss the same in more pedestrian terms, in this article; see North for the more rigorous treatment (and for other knowledge-rich nuggets).
The Business Analyst is a Role
A quick aside before we dive in, today. While I keep referring to "The Business Analyst", I'm not saying that there must be a separate person on the team who's title starts with a "B" and ends with a "t". I'm saying that there's a mindset/perspective embodied in a role known as "the Business Analyst." So, when I say, "The Business Analyst" I simply mean to say, "whoever is doing this work, with the BA mindset active."

Principles for Writing Cucumber Features

Write Features as if you were the Business Analyst

One common mistake when getting started with Cucumber is to include steps that are expressed in terms of underlying technical details. In fact, the Cucumber adaptor for Rails (via the cucumber-rails gem.) includes a number of step definitions (within web_steps.rb) that specifically call for xpath or CSS selectors. This is a bit unfortunate because to realize the total value of your Features, they need to be expressed in a way that survives changes in underlying implementation details.
Don't Be Misled By OOTB Steps
This is not to say the Business Analyst should be willfully ignorant of the implementations; quite the contrary. For example, cucumber-rails's web_steps.rb comes with a step definition that matches the pattern "Then I am on <page-name>". You might be tempted to include this in your Scenario to verify that clicking on a navigation element brought you to the correct page. However, the implementation of that step simply checks to verify that the URI of the current page matches that of the targeted page. But, if you have forgotten to define a route for that page, the URI will still match, but Rails will render the "Route Error" page. It's more correct to check for, say, the title of the page or some element that would only show up if the correct page was rendered.
The balance, lies in the perspective of the Business Analyst. Remember, this is the role that must understand both sides of the equation: the requirements as they are motivated by business concerns and the implementation details so far that they influence what's feasible to do with this system. It's from this position that one can write Features that are likely to last: informed by technical details and articulated in terms of elements of the User Interface.

Write Features in terms of the visible User Interface

A related principle is to articulate the Feature steps in terms of elements of the User Interface (UI). Terms like "buttons", "pull-downs", "title", "navigation bar", etc. are about as technical as one should get without risking losing the Product Owner who's expertise is in the business, not software development.
In practice it is helpful to not just agree on what the individual widgets are called, but to also develop an agreed-upon vocabulary for common parts of the page. Common parts include:
  • Header and Footer
  • Navigation Bar
  • Status Message Area
  • Main Body
There will also be elements that are repeated across your application. For example, if there is a heavy social networking component to your webapp, perhaps the set of "follow" buttons (e.g. reporting to your onlines friends that you've just bought the hip new new Thing... we'd present a series of buttons for Facebook like/share, Twitter, etc.)... it will facilitate language if you coin a term for that set of social networking controls: "Like Button Bar"? "Social Clicks"? Whatever you choose, it's the job of the Business Analyst to ensure that a shared vocabulary develops.
By sticking to the UI, your Features will be both concrete (more easily implementable) and better understood by both parties (i.e. Product Owner and Engineering). By developing a meaning-rich vocabulary (and implementing Features in these terms is what facilitates this shared semantic), you can bring the two sides of the team closer together. It means that misunderstandings can bubble-up faster (remember, the sooner you discover a defect, the cheaper it is to fix it, generally).
For the users of the program, the UI is the software. Keep Feature Steps in terms of the UI.

Stay DRY

DRY is an acronym for "Do not Repeat Yourself" (see Wikipedia article for a deeper explanation) and it applies just as well to Feature Step writing as it does to software coding. The essence of this advice is to take care to minimize the amount of overlap between Features and their Scenarios. This is another example of where it pays significantly for the Business Analyst to have a working understanding of how the software is functioning. If you have already written a Feature+Scenario that exercises selecting an item from a list, no need to repeat verifying that behavior in another Feature+Scenario.
Features as Functional Tests
This become especially poignant as the project grows in size. One of the enemies of Continuous Integration (CI) is long running builds.
Long running CI builds create all kinds of problems: they lengthen the commit dance which slows down disciplined developers; they dampen the whole value of the Continuous nature of CI because the time window is larger and thus more developers' changes start to pile on. Builds that push past 15 minutes start to manifest these issues. So, in the interest of maintaining the value of Continuous Integration, the full build (which includes unit and functional tests) should run as fast as possible.
Quite often the longest running component of these full builds are the functional tests. Cucumber Features in the context of requirements analysis are User Stories; but Cucumber Features in the context of verification (i.e. executing the Scenarios) are, in fact, functional tests. To the extent that there is coverage overlap of Scenarios, we're duplicating computing effort (exercising the software in nearly the same way) with no added value. This waste needlessly extends the length of the overall build. In this way, Features written in an unskillful way, endanger the value of the CI-build -- a cornerstone of Agile software development.
The Business Analyst, as the author of Features and their Scenarios, is a key player in ensuring that the functional test portion of the overall build executes as quickly as possible without reducing the confidence in correctness that these tests bring.

Practices for the Writing Cucumber Features

With those two guidelines in mind:
  • Write Features with the Business Analyst Mindset;
  • Keep Features DRY.
Here are some concrete techniques that I have found to work in the field.

Create and maintain a lexicon of Cucumber Steps

Create a page in the project wiki (in the Requirements section) to maintain a dictionary of Cucumber Steps. Organize the page into sections including:
  • a section for common steps;
  • a section for each major feature set;
  • a glossary of Domain and Visible Software Terms.
Within the Step definition sections, include a table with the following columns:
  • Template -- the generic form of the Step, with clear typographic notation to indicate the "variables" in the Step.
  • Meaning -- description of what the Step does. Include a clear definition for each "variable" and the expected set of values.
  • Examples -- like all technical documentation, nothing makes it clearer than a reasonable example. Unless it is absolutely obvious how one would write a step, include at least one example.
In essence, this page codifies of the shared vocabulary of the project's requirements. The Product Owner (through the Business Analyst) contributes Domain-specific terms and their bias to the solution. The Engineering team (again, through the Business Analyst) contributes names of widgets and other UI terms.
This page acts as a palette while writing new Feature+Scenarios. The goal should be to identify reusable steps (and often refactor an exist set of steps into a more generic one, especially in the early days of the project) and write new Feature Step using the existing vocabulary as much as possible. Don't torture the language, just look for reuse. At the same time, new Features bring new UI interactions and it is through writing Features that new kinds of Steps are discovered. As they are, add them to the page.
When we look at this documentation from an Engineering perspective, as we delve into how to implement Cucumber Feature Steps), we will see how technical concerns have an equal influence over the shape of this lexicon. In the end, however, this document sits squarely in the Business Analyst's area of concern.

Marry an Issue Tracking Ticket with a Cucumber Feature to Form Each User Story

A User Story, in its best incarnation, is a token to remind the team of an understanding around a specific requirement. In practice, there are two parts to a User Story, then:
  1. the lastest set of agreements, written down to the extent that the team need to so as to not forget.
  2. the workflow of the User Story, from conception to sign-off, tracked to help keep the whole team on the same page as to the "status" of this feature.
One way to handle these two aspects of the User Story is to create a Ticket in a flexible issue tracking system (I have personally used JIRA in the past and loved it...I'll share more about how Atlassian's JIRA+Confluence combo is a killer solution to most software development efforts) and to link that Ticket to the .feature file checked into the VCS.
The role of the JIRA ticket is to track the workflow and discussion amongst the team regarding the user story. The .feature is the home for the rest of the User Story: the narrative and agreed-upon details.
"Linking", here is simply including the URL to each artifact in the other:
  • in the JIRA ticket, include the URL to the .feature file in your VCS via its web interface (both SVN and git provide these, OOTB, others may as well);
  • in the .feature file, include the URL to the JIRA ticket.

Start with Two Scenarios: Main Flow and One Exception Flow

The practical starting point for keeping Features DRY is to start with the minimal set of Scenarios for each feature. Begin with just two Scenarios: one that covers the Main Flow or "Happy Path" and one that covers what happens when there's an "exceptional case" (usually responding to invalid user input).
The key idea is to include an additional scenario only when an additional UI treatment is introduced. In the Happy Path, there is often a message that indicates "success" to the user (e.g. "Your journal entry has been recorded."). However, if the user input fails form validation, for example, then the elements that make clear to the user how to fix the input now come into play (including highlighting around text fields and the presence of the error message in the status area.
Once those mechanisms are exercised, the value of testing for all the other conditions that cause these UI mechanisms to activate is significantly reduced. Such tests are better suited as verifications against the underlying "model" code: unit tests (i.e. verifying that when the application attempts to use invalid input, the validation code catches it). Here we see, again, that the Business Analyst, the author of these functional tests, is well-served understanding how the underlying software is structured.

Next...

In this installment of our series, we've covered what life is like for the Business Analyst writing Cucumber Features. Hopefully, this description starts to illuminate the power of using Cucumber Features as the center-piece for BDD -- since they clearly sit right in the Business Analyst's purview, they are the confluence of business desire and software implementation concerns. Because Features are executed (and indeed, incorporated in the CI build), the Business Analyst looks to keep the formal Scenarios tight and DRY.
With the bryne starting to work its way in, we're ready to turn our attention to the full pickling process: how the raw requirements meet and become infused with the salty world that is software engineering. We shift our focus from the Business Analyst to the Engineering team who is writing the Ruby code to implement the Cucumber Steps...

Monday, January 10, 2011

Relishing in Perfection Part 2 -- Ingredients


Installment two of "Relishing Perfection", a series on doing BDD with Cucumber, "the right way". If you have not already, read the introduction.

Reflections

And now, some observations from having experienced BDD and Cucumber, up-close...

Feature Authoring is Iterative

Like everything else in Agile development, arriving at the "right" way to express acceptance criteria is best done organically, iteratively. There are certainly patterns to be hatched, but this is a developed capability. Expect to learn as you go and to be evolving your own Business Readable DSL.
I share a number of "low-hanging fruit" Cucumber Step "templates", in a future post.

Feature Authoring Facilitates the Requirements Conversation

Features are executable requirements. In fact, there are essentially User Stories. They are discovered, evolved and revised as such. Because Features live in the gap between the Customer and the Developer, both perspectives are required: what the Customer wants and what the Developer can deliver.

Features are Managed by the Business Analyst

We're engaged in zeroing-in on the right software solution to the business problem at hand. The traditional role that owns that quest is the Business Analyst. In a mature Behavior-Driven Development project, the Business Analyst is the perfect role to be the "Chef-in-Chief". The Business Analyst:
  • develops a solid understanding of the business domain (the problem domain); so much so that she can regularly speak for the customer in their stead.
  • also learns well what's possible in software (the solution domain); again to the point where she can often represent for the development team in their absence.
  • seeks out, captures and develops requirements; like a sleuth, hunts down missing details; brokers the best feasible solution.
  • ensures that the software developed meets not just the letter of the requirements but the spirit of the envisioned solution.
Now, in your organization, you may or may not have a title "Business Analyst" but these concerns must be managed by one or more project team members. Sometimes, the Project Manager doubles as the Business Analyst (in all but the smallest or least inventive efforts, a real mistake). Sometimes the Technical Lead also picks-up some BA responsibilities (again, this role is usually a full-time effort in itself). It's not important that there be a person with this title. It's important to have this mindset and perspective active when writing Features.
If you approach your Features as if you are the Business Analyst, you'll:
  • more naturally use language that is understood by both business folks and developers; Martin Fowler describes this as Business Readable DSL.
  • know how important it is to use language consistently and do so when writing stories and features;
  • know when the Scenarios of a Feature are "good enough" to begin development.

All the World's a Nail...

Like with any new tool, it's easy to over-apply its use. In the case with Cucumber, especially in the context of the BDD process, it is easy to get caught up in writing a ton of Scenarios. This is requirements driving development, isn't it?!
It's true: we are developing software to serve the business, not the other way around. However, the pragmatic reality is that even with a simulated browser (OOTB, Cucumber uses Capybara) these tests take time to run... far more time than targeted unit tests/specs. Here, the principle of being test-driven meets the principle of "Don't Repeat Yourself". The balance is in enough functional tests (which is really what Features are: executable Functional Specs) to give confidence that the UI is generally working, backed by unit tests/specs that ensure the underlying components operate in the larger set of expected conditions.
When we started our project, we were purely driven by Scenario development... which is not a bad place to start. Establish those habits early. It wasn't long (the second or third feature) that our CTO (with a fresh perspective on the project) observed that we were writing a lot of Scenarios... one for each validation case. When we looked closer, we realized that we were repeating a lot of setup and clicking around just to verify that the right error message was displayed...

Be Agile: Inspect and Adapt

So, we adjusted. We realized that there was little incremental value being produced with these additional Scenarios over the "Happy Path" and one "Invalid Input" pair. The important additional cases (e.g. the whole rest of the possible flavors of invalid inputs) could easily be tested (and cheaply, both in coding and execution time) in the unit test environment (in this case, RSpec).
The key is that we made sure we were not slaves to the process. This is a core value in the Agile approach: give more credence to educated and contextualized perspective than process dogma. This is part of what it means to value individuals and interactions over process.

Next...

With that, we are ready to start pickling. There are two major skill sets to develop: authoring Features themselves and developing the code for the Cucumber Steps. They feed into each other but have their own concerns as well. In the next session, I will dive in the first of these two skills, hatching principles and practices the Business Analyst should use when authoring Features in the first place.

Saturday, January 8, 2011

Relishing in Perfection: Doing BDD the Right Way with Cucumber


This last week, I ramped-off the first project I've been on using Behavior-Driven Development. It was a Rails project and so we used Cucumber. As I looked out across the web, I found a few beacons around the actual practice of BDD. There's some good stuff out there, but I couldn't find anyone that is going past the theory and sharing the tacit knowledge. This is my first attempt at starting to fill that gap.
First, I want to contextualize BDD and share some resources to get started. Then, I give some concrete observations from our project and lay out a set of principles and practices that we followed or discovered.

Advancing The State of The Craft

Dan North's Behavior-Driven Development (BDD) is proving to be another key stepping stone in our journey, as an industry, toward what Uncle Bob has called "professional craftsmanship".
One of the fundamental practices in BDD is "outside-in programming". Programming efforts are driven by attempting to meet a specification of the software's behavior as quickly as feasible. "Specification" sounds pretty rigorous, doesn't it? ...and a good specification is.
The most useful specifications come with examples that concretely express what "done" looks like. This is really important, because if you can get this concrete, you can start to have meaningful conversations between the folks driving the business and the kids writing the software. Paired with visuals (story board of wireframes ... aka "User Interaction Designs"), you can really start to have design discussions with actionable, trackable results.
In this way, examples facilitate quality communication -- the means by which intention, vision, and detail are shared. Good stuff! But wait, there's more...
What if you could express your specifications as examples in a concrete enough way that you could automate the execution of them? Suddenly, the "definition of done" is more firm (if the automated example doesn't run successfully, you're not done). And if the examples are abstract enough, we can expect to execute them not just when the software is developed, but also as part of a regression suite. Each time you run it, reinforcing your confidence that "everything is still working." Wow! That's a lot of value being realized. But can it be done?
We're really asking...
Is it possible to express acceptance criteria as examples that they:
  • are written in language understood by both business and software people?
  • are meaningful enough to act as a catalyst for converging on the "right" software solution to the business problem at hand?
  • are concrete enough to be executed by a computer?
  • are abstract enough that one can expect them to function long after that part of the software is complete?
That's a tall order... but that's exactly what BDD is driving towards. If we have any hope of getting there, we'd have to see:
  1. exactly how these kinds of examples are written;
  2. precisely how they are made executable with a minimal effort;
  3. and how these practices fit within the software development process.
Thankfully a huge chunk of this ground has been expertly covered. David Chelimsky covers the very foundations of this discussion in The RSpec Book: Behaviour-Driven Development with RSpec, Cucumber, and Friends (with significant contributions from Dave Astels (author, RSpec contributor and BDD evangelist), Zach Dennis, Aslak Helles√ły (original author of Cucumber), Bryan Helmkamp, and Dan North (credited with the term BDD and write RSpec's “Story runner," the predecesor to Cucumber). As Robert Martin eloquently explains in the forward (posted right on the PragProg page for the book), the title of the book is misleading; it would be more aptly titled "BDD in Action".
Before he published his book, Chelimsky developed a presentation:
Integration Testing in Ruby with RSpec’s Story Automation Framework" which contains enough detail that it can be read on its own.
What Chelimsky's book covers is how Behavior-Driven Design can be realized in the Ruby world (and specifically when developing web applications with Ruby on Rails). As the title suggests, his book walks you through a mini website development project employing the BDD approach to identify and using Cucumber to write the very kinds of executable Examples we're talking about here.
While not the first of its kind, Cucumber has in the past year become the tool of choice for Rails shops that are adopting BDD
(Alex Handy wrote a fine article about Cucumber and included its brief history, titled "Peeling Cucumber").
What's needed, then, are reports from the field: empirical data and observations that fill-in the important details on how a project team realizes the benefits of BDD.
Having completed my first such effort, I'm no expert. But I have done it and definitely have some insight to share. In the next weeks, I will be unspooling a series that makes an attempt to being to fill this gap:
  • sharing observations about how Feature authoring unfolds, in practice;
  • enumerating principles and practices around writing Features and Scenarios;
  • discuss similar guidelines for smartly implementing your Cucumber Steps;
  • suggesting a set of common templates and features that you should include in your Example automation suite.
If you're getting started with BDD and Cucumber, yourself, I hope you'll join me. Better yet, I hope you'll engage me in the conversation and share YOUR experiences: we're much smarter together than running solo. Let's relish!

Next...

Before we charge into the kitchen and start mixing it up, let's take a quick survey of the Ingredients...

References:

Professional Craftsmen

Behavior-Driven Development

Cucumber