Testing browsers concurrently
Over the last few months I’ve been using Cucumber to write Behaviour-Driven-Development tests for my web projects. The tests I’ve been writing have been fairly simple and sequential. The tests would, for example, authenticate a user, perform some action and then log out. Only one user was required to perform an action at a single time. If two users were required to interact then the test would log out as the first user and authenticate with the second. The application could wait for the other user to re-appear before delivering it’s action. I’ve recently started writing some BOSH applications which require both users to be logged in at the same time. Testing these require two browsers to be open concurrently.
A very basic sequential scenario
This is an example of a simple sequential test which only requires a single browser session.
Scenario: A message should be sent to another users
Given I am on the index page
When I enter a random username into "register_username"
And I enter a random password into "register_password"
And I enter a valid email address into "register_email"
When I click "Register"
Then I should see the text "Registration successful"
As you can see from above it registers a user with the web site. This is an easy scenario to programme because it only involves one user. Scenarios can become more complicated when it involves interactions with other users:
Scenario: Poke another user
Given I have a user called Mary
Given I have a user called Dave
When I log in as Dave
And I am on the messages page
When I click "Poke" Mary
Then I should see the text "You have poked Mary"
Then I log out
When I log in as Mary
Then I should see the text "Dave has poked you"
In this scenario we are testing “Poke” functionality. You can see from above that for the user “Mary” to receive her “Poke” the test has to log out of “Dave” and re-authenticate. Ideally we would like to be able to test receiving the message immediately. This would remove this scenario’s dependency on the log out steps, which could be another scenario.
We can also identify that only lines 4,5,6,7 and 10 are related to the scenario outlined on line 1. The rest is either setting the scene or fluff required for the scenario to pass. The fluff doesn’t directly correspond to a real user, so arguably shouldn’t be there.
A more concurrent scenario
The following is an scenario written without the unnecessary log out steps.
Scenario: Poke another user
Given there is a user called Mary
Given there is a user called Dave
Given "Dave" is logged in
Given "Mary" is logged in
Given "Dave" is on the messages page
When "Dave" clicks "Poke" "Mary"
Then "Dave" should see the text "You have poked Mary"
And "Mary" should see the text "Dave has poked you"
In the above scenarios we no longer have any steps which starting with I, each step refers to a users who is performing the action. From this scenario you can see that the scene for the scenario is set up with the first 5 Given statements. This allows for the When, Then, and And statements to describe only the actions required for the scenario, not anything required for setup or the fluff described earlier.
Creating some more complex rules
We have to modify our steps to match the user’s name at the beginning:
1 When /^\"?(I|[^\"]*)\"? clicks? \"([^\"]*)\" \"([^\"]*)\"$/ do |who, what, to| 2 # Implementation pending 3 pending 4 end
The above rule matches line 7 from the previous scenario. You can see that it involves a more complex regular expression to extract which user is undertaking the action.
A choice browsers
The Cucumber Watir examples shows us how to choose between Firewatir, Celerity and Watir’s Internet Explorer and Safari support.
1 if ENV['FIREWATIR'] 2 require 'firewatir' 3 Browser = FireWatir::Firefox 4 else 5 case RUBY_PLATFORM 6 when /darwin/ 7 require 'safariwatir' 8 Browser = Watir::Safari 9 when /win32|mingw/ 10 require 'watir' 11 Browser = Watir::IE 12 when /java/ 13 require 'celerity' 14 Browser = Celerity::Browser 15 else 16 raise "This platform is not supported (#{PLATFORM})" 17 end 18 end 19 20 # "before all" 21 browser = Browser.new 22 23 Before do 24 @browser = browser 25 end 26 27 # "after all" 28 at_exit do 29 browser.close 30 end
The above code selects a Watir variation based on the available libraries. Notice that it creates a Browser object which implements the available Watir variation. The Browser object is then instantiated and made available during the step definitions.
To cope with multiple browsers I made some modifications to the Cucumber example above. Firstly I change the browser and @browser to be a hash of Browser instances.
1 browser_instance = {} 2 browser_port_count = 6429 3 4 Before do 5 @browser_instance = browser_instance 6 @browser_port_count = browser_port_count 7 end 8 9 # "after all" 10 at_exit do 11 browser_instance.keys.each do |key| 12 browser_instance[key].close 13 end 14 end
The key for the browser_instance hash would be the user’s name provided by the step definition. To aid the look up from the step definition I wrote the following helper:
1 def get_browser(who) 2 if not @browser_instance.has_key? who 3 @browser_instance[who] = Browser.new 4 @browser_port_count = @browser_port_count + 1 5 end 6 return @browser_instance[who] 7 end
You can see from the above code that whenever the key does not exist in the browser_instance Hash a new Browser instance is created.
The above code will not work on all implementations of Watir. We now need to make our Cucumber tests a little more specific to the Watir variants.
So an example step definition using the code outlined above is:
1 Then /^\"?(I|[^\"]*)\"? should see the text "([^\"]*)"$/ do |who, what| 2 if not get_browser(who).text.include? what 3 fail get_browser(who).text 4 end 5 end
Celerity
Celerity is an implementation of the Watir API using HTMLUnit, a headless Java-based web browser. I’ve used Celerity to work with multiple browser instances, but this also presents some problems.
Celerity will allow you to create multiple instances of the browser, but will not allow you to attach a viewer to the same port. So if you need to view your tests running you’ll need to specify a viewing port:
1 @browser_instance[who] = Browser.new({:viewer => "127.0.0.1:#{@browser_port_count}"})
It’s worth mentioning here some other useful options to Celerity:
:log_levelchanges the amount of logging you’ll see from Celerity. I have this set to:all.:javascript_exceptionswill raise any Javascript errors:resynchronizewill resynchronize any Ajax calls. You’ll need to pause Celerity to wait for any Ajax responses. There is more information on handling Ajax requests on the Celerity Wiki.
Before running your tests you’ll now need to run a viewer on different ports. This can be done with the following command:
QT_CELERITY_VIEWER_PORT=6430 ./QtCelerityViewer &
Repeat the command incrementing the port number for every new user you want created.
Firefox
Unfortunately Firewatir does not support multiple instances. Firewatir works by sending commands to the a ‘jssh’ extension installed in Firefox. jssh listens port 9997, and it is not possible to change the port number unless you install coderr’s modifications.
Once you have installed coderr’s modifications you can then create Firefox instances using the following code:
1 @browser_instance[who] = FireWatir::Firefox.new(:port => @browser_port_count)
Another problem with Firewatir is that it does not support Firefox 3.6 on Linux, my choice of development platform. Uninstalling the Ubuntu package for Firefox I installed the generic Linux Firefox build from Mozilla.org.