Advanced automated testing using AI

Business case

When a customer website is critical for his business and a lot of users are interacting with the website. Even with a content review process some non-compliant contents or issues may be missed. And above that, sometime the pictures have less attention then the text.
But how could we test picture content?

Quality objectives

Here are some non-exhaustive objectives. What are the quality objectives on the pictures displayed on the website?

  1. Picture object are relevant with the description.
  2. Avoid certain type of object content in pictures.
  3. The picture content doesn’t contain unauthorised logo.
  4. People on the picture are not happy enough.
  5. The picture colors are not following the graphical chart.
  6. Misspell on title in the picture.

What are the technology solutions to achieve this goal?

There could be several solutions, but let’s narrow a bit to one particular solution that we are working on it.

We have started from a test factory based on Serenity, Cucumber and Selenium.

Depending on the test scenario, when it is required to validate a picture, we send the picture to an service that allows to validate a picture content based of acceptance criteria like defined in the quality objectives above.

How this picture validation works?

The service is based on Google Vision Api. This Api is able to identify objects on picture. But this Api is not perfect and only provide relevancy percentage on different options. In order or be accurate, the AI need some context and that’s what we provide him to get a better relevancy and have a final answer.

This service solution is not free. Google is charging any transaction to the Api. So some caching process need also be taken in account when the same picture with identical content are analysed several time.

This Google API is using the machine learning and and his big data. This Api is the most popular service even used by the US Air Force to identify objects on picture taken by drones or satellite. Each day it becomes more and more “clever”. That’s why we have chosen this Api to build our service for picture object identification.

The object identification could be items like human constructions, human faces behaviours, animal and vegetal identification and much more.

AI or not AI

Some AI “expert” could be not agree with this, but we want to keep this simple. For us AI means “a branch of computer science dealing with the simulation of intelligent behavior in computers” (merriam-webster). Google Vision API is based on Google Big Data and Machine Learning to be able to simulate this human behaviour. So for that reason we consider as AI.

AI service release

We are still working on this service. This service using AI is in alpha stage and we are preparing for Q1 2019 an initial release. We are in an optimisation and testing phase. Stay tuned for more info on twitter @fanaticaltest.