Smart Automated Visual Testing
Ever since Applitools came onto the visual UI testing market in 2014 we have all had our eyes opened (pun intended) to the benefits of smart image comparison testing to enhance our web and mobile app test automation. Those smart Israeli’s not only developed a great product and but a very efficient “pay per check billing machine”. How many times a day do you kick off a test run, only to realise that you have forgotten to make a test change that you intended. Each time you do this with “their eyes open” it makes “your eyes bleed”, when you add up your test automation costs at the end of the month.
When automated visual UI testing was a nice to have, you could “ration” when you applied this type of testing. But how else do you check the layout, size or location of your increasingly visual web and mobile applications? The move to responsive web applications that change layout based on viewport size makes this testing even more necessary. So now we find ourselves stuck with the reality that visual checking has moved from a nice to do, to an absolutely MUST do.
Free & smart & intelligent image comparison service
We recently came across this excellent free visual checking capability that perfectly fitted our automated visual testing needs. It is included in the increasingly brilliant open source test automation framework WebdriverIO and was developed by the great Wim Selles.
So let me explain what we were trying to do and why this open source beauty helped us solve our testing problem. Our small team of developers have been working on a new cloud based version of our AI-driven test automation tool, Vitaq AI. We developed our AI-assisted test automation engine to help QA and Dev testers rapidly release their application updates. Our tool monitors, measures and analyses functional coverage in a cloud database. It feeds this into our AI engine and uses this machine learning to work out what tests to automatically generate next to extend your testing coverage. Our patented technique is proven to find unknown unknowns. The types of bugs that can easily be released to your customers!
Functional coverage relates to the behaviour of your application. Just as a clarifying note, it is different to code coverage which relates to the structure of your application code. Our functional coverage viewer is very visual and pops up in a new browser tab when a user wants to see their test coverage progress, as shown below.
Block Out custom regions to visually check ONLY what you want
If we did a simple screen compare on the above Vitaq functional coverage viewer tab our tests would fail from one day to the next. This is because each time we run the test, we get a different unique test name based on the dates and versions such as “globalsAllSequenceCoverageCountTest_20210305_111150”.
This free open source image comparison service is super smart and flexible. Not only does it automatically create your baseline images (those that you will be visually checking against) it allows you to easily specify which parts of your web or mobile app screen you do not want to include in the image comparison check. All I need to do is tell the image comparison service what CSS locator (or XPath) I want to block out as shown below in the code.
The CSS selectlor #menu_button_test in the removeElements configuration above is the drop down Test menu in my coverage viewer web page. So now I get a visual check which “blocks out” this part of the screen as shown below.
Why pay when you can get it for free?
Et voila. This smart automated visual testing is really easy to use and extremely robust. We use it with webdriverIO and chromedriver for our dev testing and then with Selenium grid for our QA cross browser testing.
Don’t you just love how webdriverIO just gets better and better. The new V7 release is fabulous and with Selenium 4 just coming onto the scene it is an absolute joy. Using these powerful opensource frameworks with hundreds of super smart contributors is like having a massive development team all working to help us automate our tests.
Keep up the great work peeps. We love standing on the shoulders of giants because we can get so much more testing done.