intel / p3-analysis-library

A library simplifying the collection and interpretation of P3 data.
https://intel.github.io/p3-analysis-library/
MIT License
7 stars 10 forks source link

Improve testing of gallery pipeline #64

Open Pennycook opened 3 weeks ago

Pennycook commented 3 weeks ago

Feature/behavior summary

37 introduced a GitHub Action to run the gallery, which will make it easier for us to inspect the images generated after certain changes. If we could find a way to automatically identify whether any of the generated images have meaningfully changed, that would save us the hassle of having to check the images manually.

Request attributes

Related issues

No response

Solution description

@laserkelvin suggested that we might be able to use a "perceptual hash" to compare the generated graphs against previously generated graphs, as it may be more robust than something like MD5/SHA-512 if something moves a little bit.

Additional notes

We probably wouldn't want the action to fail just because an image changed (since it may be deliberate). Some sort of warning directing our attention to a files that should be double-checked by a human would be good enough.