As the title of this document suggests, this document concerns itself with testing. However, please note that the document lists stories about the usage of the system. These stories, taken together, outline the requirements that the system is designed to fulfill. Therefore, this document addresses requirements as well as testing. After all, requirements and testing are fundamentally related, since the requirements of any system define the boundary between correct and incorrect behavior on the part of the system, and testing provides some means to judge whether the system meets the requirements when it is exercised.
When registering a person as an editor, the administrator can (must?) set the password for the new editor.
The automated test case 20_basic_sanity/010_editors/010_create.sel provides a simple test of the requirement that the administrator must be able to register a person so that the system can know that person as an editor.
This test case depends on the setup procedure having been run, which logs the session in as the administrator. The setup procedure looks like a suite of two test cases under the directory 10_setup . The tests are arranged such that if you allow the test runner to run all the tests at once, it will run the setup suite right before it runs 20_basic_sanity/010_editors/010_create.sel , thus fulfilling the latter's precondition of having the session already logged in as the administrator.
The automated test case 20_basic_sanity/010_editors/020_change_password provides a simple test of this ability. It depends on the prior test case for its preconditions, those being that the session is logged in as the administrator and that the editor whose password we intend to change has already been described to the system as an editor.
The automated test case 20_basic_sanity/010_editors/030_delete provides a simple test of this ability. It depends on earlier test cases for its preconditions, those being that the session is logged in as the administrator and that the editor that we intend to remove has already been entered into the system.
The automated test case 20_basic_sanity/010_editors/060_sign_in.sel tests this capability. The test depends on earlier tests for having set up the system to meet the precondition that the editor as whom we intend to sign in has already been entered in the system (and entered more recently than the last time removed).
This test case establishes a precondition on which all the subsequent tests in the "basic sanity test" suite depend, namely, that the session is logged in as an editor.
The automated test case, test/selenium/20_basic_sanity/020_form_submissions/010_change_notification_addrs_via_form_submissions_tab.html demonstrates this capability.
Currently the test plan does not offer any stories for more thorough testing of Editors.
Currently the test plan does not offer any stories for more thorough testing of Form Submissions.
Currently the test plan does not offer any stories for more thorough testing of Files.
Currently the test plan does not offer any stories for more thorough testing of Images.
Currently the test plan does not offer any stories for more thorough testing of Pages.
Currently the test plan does not offer any stories for more thorough testing of Dashboard.
The degree of automation of the automated tests does not suffice to relieve you of typing more than one command. Several steps are required for setting up and running the tests. However, the number of manual steps required to get the computer to do the automated tests remains fixed, regardless of how many test cases we include in the suite of automated tests.
First, it is necessary to clear out the database that you use for testing. To do that, you can position yourself in the root directory of a copy of the system (in which you have already set up config/database.yml and for which you have already created the testing database) and enter the following commands in the shell:
rake db:migrate VERSION=0 RAILS_ENV=test
rm -rf public/uploaded_*
rake db:migrate RAILS_ENV=test
rake application:initialize RAILS_ENV=test
As I write, the attachment mechanism seems to be failing in the test environment. Just to get going before bothering to solve that problem, let's try using the the development environment instead of the test environment. If you do this, make sure that you are doing this in a copy of the system where you don't value any of the data you have in your development database. This procedure will discard whatever you may have there.
rake VERSION=0 db:migrate
rm -rf public/uploaded_*
rake db:migrate
rake application:initialize
The above procedure may need refinement if, because of the evolution of the Rails framework, any of the migrations start failing to work, or if the amount of time it takes to undo and redo the migrations becomes annoying.
In any event, the next step is to start up a server process for the testing. For the following example, I use port 3011. You might want to be in the habit of using distinct port numbers for testing and development. If you are sharing a development environment with other programmers, you should negotiate with them a regime for avoiding clashing with each other on port numbers.
script/server -e test -p 3011
If we're not specifying the test environment, it's
script/server -p 3011
Once that starts, you want to aim a browser at "/selenium" within the URI region served by the server process you just started.
If you are browsing at a computer other than the one running your server, and if the hostname of the latter were, for example, "ccts-2008-11-25", and sticking with the same port number as above then the following would be the URI to put in the address field of your schnauzer:
http://ccts-2008-11-25:3011/selenium
On the other hand, if you are sitting in front of the testing machine, you have to use "localhost" instead for the hostname, so the URI in that case comes out like this:
http://localhost:3011/selenium
To be continued . . .
The vehicle for developing test scenarios is Firefox. You can test in any browser (and probably should test in all of them from time to time); however, only in Firefox do we have (as of this writing) a tool available to help you record an interaction.
To install the tool in your personal configuration of Firefox on any given computer you use, the first step is to choose in the menu, Tools -> Add-ons. Make sure the "Selenium IDE" extension is installed and enabled ("IDE" stands for "Interactive Development Environment"). It is available in the selection of add-ons that Firefox offers for installation. You shouldn't have to worry about the location from which to load it.
When you want to record a testing scenario, begin by opening the browser against a testing server that you will have started the same way as though you were going to run tests, as directed in the previous section of this document.
Take whatever actions are necessary to get your database into the state appropriate for the start of the test case you want to compose. Aim your browser at the base URL of the server you are testing.
For further information about running the "Selenium IDE", consult http://wiki.openqa.org/display/SIDE/Getting+Started+with+Selenium+IDE .
When you want to start recording the scenario, choose Tools -> Selenium IDE.
The Selenium "IDE" will come up as a little window of its own. I find that it usually seems to come up with recording enabled. The big red dot on the right near the top functions in effect as the "record" button. If you hover your mouse pointer over it, you should see a hint come up that will state whether recording is on.
At this point, for most of the kinds of test scenarios you will be recording, it may be a good idea to load the "Base URL" field near the top of the Selenium IDE windowlet with the same URL that would in the browser get you to the home page of the server under test.
Then usually the next thing to do is to aim the browser at "/admin" relative to the base URL. Initially, the display, in the Selenium IDE, of the steps of the test scenario you are constructing, will not show the "open /admin/" command. However, it will show up after the next step.
The usual next thing to do is to click on the button in the CMS for the area you intend to work in, one of Pages, Images, Form Submissions, Editors, or whatever other areas get added in the future.
So then just continue interacting with the web site under test. The Selenium IDE windowlet should reflect your actions in its growing table.
To make the test scenario into a test, you want to include one or more commands to make it verify that something that should be on the screen really made it there. A fairly reasonable target for a check of this kind, in many cases, would be the confirmation message the web site puts up, for example, "Successfully updated Contact Us". To append such a command to the scenario, sweep your cursor over the confirmation message, right click, and choose "verifyTextPresent" from the pop-up menu.
Next you will want to save the test script so you can incorporate it into the suite of tests stored with the source code of the CMS.
. The selenium_on_rails plugin interprets the test scenarios. It finds them in the test/selenium directory and its descendent directories and presents them when you follow the directions in the previous appendix. For further information on the plugin, you can refer to vendor/plugins/selenium-on-rails/README.md relative to the project root. However, some of the statements appearing there may deserve a grain of salt. In particular, it says that you can export test cases from the Selenium IDE in .rsel format as accepted by the plugin, and I have not found a way to do that without manually editing the files.
| Term | Definition |
|---|---|
| CMS | Content-Management System |
| IDE | Interactive Development Environment |