Visual regressions
All Pulsar helpers and most components have a related 'fixture' file which is used to test expected markup by rendering with both Twig and Twig.js, we also use these fixtures to visually test for changes between releases with the BBC's Wraith testing tool. Changes are visually highlighted in blue.
The fixtures are located at /tests/unit/Jadu/Pulsar/Twig/Macro/, the files which are tested by this tool are the ones ending in *.html.twig.
Visual regression testing allows developers to test their CSS changes do not adversely affect unrelated components by creating 'diff' images of any elements which have changed.
Initial setup
You will need to install the following local dependencies, Pulsar's make command will attempt to install them for you.
- Homebrew
- RVM
- Ruby (version 2.3.0 works for us)
- ImageMagick
- PhantomJS
- Wraith
Testing for differences
To test for changes, Wraith will take a second set of screenshots and visually compare them with the baseline, generating a report at the end which will list any differences for you to review. Run the following command:
grunt wraith
This will screenshot of all the fixtures listed in the wraith.yml file and compares them against the baseline images in /shots_history.
If any images differ from the baseline, a new diff image will also be added to /shots_history directory.
Recreating the baseline
A baseline set of screenshots is already included in the Pulsar repository, but if you need to update them, run:
grunt wraith-update
This task scans the fixtures directory and updates the wraith.yml file in the pulsar root directory, then takes screenshots of all fixtures, placing them in the /shots_history directory.
This establishes a the baseline set of screenshots for the grunt wraith task to test against.
Reviewing the results
When the grunt wraith task finishes, a gallery of screenshots will be created for you to review at http://192.168.13.37/shots/gallery.html. Shots with differences will be at the top of this gallery. (Pro tip: ⌘+click on the link that is shown in your terminal after the task finishes).
