Advanced automated testing using AI

Business case

When a customer website is critical for his business and a lot of users are interacting with the website. Even with a content review process some non-compliant contents or issues may be missed. And above that, sometime the pictures have less attention then the text.
But how could we test picture content?

Quality objectives

Here are some non-exhaustive objectives. What are the quality objectives on the pictures displayed on the website?

  1. Picture object are relevant with the description.
  2. Avoid certain type of object content in pictures.
  3. The picture content doesn’t contain unauthorised logo.
  4. People on the picture are not happy enough.
  5. The picture colors are not following the graphical chart.
  6. Misspell on title in the picture.

What are the technology solutions to achieve this goal?

There could be several solutions, but let’s narrow a bit to one particular solution that we are working on it.

We have started from a test factory based on Serenity, Cucumber and Selenium.

Depending on the test scenario, when it is required to validate a picture, we send the picture to an service that allows to validate a picture content based of acceptance criteria like defined in the quality objectives above.

How this picture validation works?

The service is based on Google Vision Api. This Api is able to identify objects on picture. But this Api is not perfect and only provide relevancy percentage on different options. In order or be accurate, the AI need some context and that’s what we provide him to get a better relevancy and have a final answer.

This service solution is not free. Google is charging any transaction to the Api. So some caching process need also be taken in account when the same picture with identical content are analysed several time.

This Google API is using the machine learning and and his big data. This Api is the most popular service even used by the US Air Force to identify objects on picture taken by drones or satellite. Each day it becomes more and more “clever”. That’s why we have chosen this Api to build our service for picture object identification.

The object identification could be items like human constructions, human faces behaviours, animal and vegetal identification and much more.

AI or not AI

Some AI “expert” could be not agree with this, but we want to keep this simple. For us AI means “a branch of computer science dealing with the simulation of intelligent behavior in computers” (merriam-webster). Google Vision API is based on Google Big Data and Machine Learning to be able to simulate this human behaviour. So for that reason we consider as AI.

AI service release

We are still working on this service. This service using AI is in alpha stage and we are preparing for Q1 2019 an initial release. We are in an optimisation and testing phase. Stay tuned for more info on twitter @fanaticaltest.

Serenity BDD in FanaticalTest Web Test Factory

We are super exited to release today a new web test factory that allows you to create functional automated test of any web application or website. This factory is open to anyone and anyone could contribute.

What’s new in 2.0?

We have remove our own framework and we decide to adopt Serenity BDD framework. We have decide to go with Serenity BDD in order to focus on test delivery and not reinvent the wheel. Also this framework has so many contributors that allows to keep it-self up-to-date.

Also the factory is delivered with a Gradle including some build tasks examples to manage several level of tests.

Last but not least, we provide a docker container for selenium agent in order to have a full selenium grid.

Getting started quickly

  1. Clone or fork the test factory.
  2. Open the project in your favorite Java IDE.
  3. In the terminal window, go to the projects root and run the selenium agent :
    docker-compose up -d
  4. Run the vnc to open the agent in order to see live the test:
    vnc://127.0.0.1
  5. Run the smoke test:
    gradle clean smokeTest
  6. When the test is completed, run in the browser [project-root]/target/site/serenity/index.html and you should see a report similar to this example.