Screenshot Testing with Chromatic

May 21, 2018
bubble chart

Is this bubble chart correct?

Answering this question with unit tests requires breaking it down into several smaller questions:

  • Is the method responsible for calculating bubble size working as expected?
  • Does VictoryScatter render the correct number of Point components?
  • Does each Point component receive the correct props?
  • Does each Point component render the correct SVG path?
    • Is the path circular?
    • Is the path centered around the correct coordinate?
    • Is the path styled correctly?
  • Are the axes correctly positioned?
  • Do the axes display correct tick labels for the given data? And so on...

No matter how many unit test you write for a given component, one nagging, unanswerable question always remains- What have I forgotten to test?

Snapshot testing can help answer this question, but its answers are often unhelpful, particularly when testing the output of SVG-heavy components. Diffs can be verbose, and border on unreadable, especially when components render complex paths. Worse, these test can be extremely brittle, with changes in things like attribute order causing spurious test failures.

Screenshot testing is a perfect match for an SVG-based charting library like Victory. Visual diffs make changes apparent and unmistakable. Victory relies on Storybook and Chromatic for automated screenshot testing. We maintain a library of stories that describe Victory components both in isolation, and composed in complex charts. Chromatic builds this library of stories with every pull request, and compares it against a set of accepted stories on the master branch. Any differences trigger a notification on the pull request:

pr-pending

The Chromatic dashboard gives an overview of everything that has changed.

dashboard

And, best of all, Chromatic highlights visual differences in individual stories. In the bubble chart below, each bubble is slightly too large. The difference is very slight, but instantly apparent when highlighted in bright green.

diff

Screenshot testing with Chromatic has been an extremely useful and confidence-inspiring addition to Victory's test suite. Chromatic also lends itself to collaboration and open source projects. Anyone can explore Victory's library of visual tests and sumbit pull requests to add more.

Related Posts

Victory Native Turns 40

November 17, 2023
Victory Native XL is a ground-up rewrite of Victory Native that diverges from the Victory Web API to leverage modern tooling in the React Native space to offer peak performance and flexibility.

Empowering Users: Developing Accessible Mobile Apps using React Native

June 26, 2023
Robust technical accessibility strategies and best practices we implemented for a mobile voting application using React Native.

Build a Text Summarizer in 100 Lines of Code with Vercel AI SDK

June 16, 2023
Vercel recently announced their AI SDK. I wanted to see just how easy it was to get up and running with this Vercel AI SDK and build something simple, but with some potential usefulness.