Image Comparisons

Comparing images

❌ Don't: Implement image comparison features before reading the documentation

Image comparison is not a straight-forward process. Implementing an image comparison submodule is not encouraged.

✔️ Do: Use the Image Processor

QualityMate offers a utility called Image Processor to easily compare images. You can read more about it on Image Processor.

Visual equivalence

❌ Don't: Do a full-screen screenshot

User interfaces change often. Full-screen screenshots will have to get changed as the UI changes, making your tests fragile.

✔️ Do: Avoid image comparison if possible

Image comparison should be a last resource. If possible, use the controls' properties to perform visual equivalence tests because they are more performant and resilient.

✔️ Do: Validate the controls relevant to the test

If you must compare images, try to limit the image processing to the controls relevant to the test as opposed to the entire screen.

Saving images on tests

❌ Don't: Save images on disk temporarily for comparisons

Saving a control's image to disk during your test adds unnecessary complexity and cleanup. The only image which needs to be saved is your baseline.

using(Bitmap tempImage = control.GetScreenshot())
{
    // this is unnecessary
    tempImage.Save("tempImage.bmp", ImageFormat.Bmp);
    return ImageProcessor.AreEqual(baselineImagePath, "tempImage.bmp");
}

✔️ Do: Use the control's image directly

When you ask for a control's screenshot it is returned as a Bitmap. Use that Bitmap directly to perform your comparison.

using(Bitmap tempImage = control.GetScreenshot())
{
    return ImageProcessor.AreEqual(baselineImagePath, tempImage);
}

Image formats

❌ Don't: Compare images with different formats (extensions)

Comparing images with different formats is less robust, because each format represents images differently.

✔️ Do: Use the easiest format to work with

Some formats are easier to compare and maintain than others. As a general rule of thumb, we recommend using Bitmap (BMP). To learn more about image formats, read this article on them.

Using AreEquals

❌ Don't: Use images with different resolutions

Using ImageProcessor.AreEqual will likely yield incorrect results unless your images have the same dimensions and are very similar.

//this will likely fail if:
// - image1 has different dimensions to image2
// - image1 is significantly different on a
//   pixel-by-pixel basis to image 2

ImageProcessor.AreEqual(image1, image2);

✔️ Do: Ensure the images are very similar before committing to them

If your image comparison fails and you're not sure why, perform these steps to debug it:

• Save both images to disk and perform a visual validation to make sure they're equal.

• Measure them to ensure that both images have exactly the same dimensions. If that's not feasible then consider using ImageProcessor.Contains instead.

• Make sure both images share the same image format, preferably .bmp.