search

matplotlib / pytest-mpl

A pytest plugin to facilitate image comparison for Matplotlib figures
Other
240 stars 46 forks source link

Started implementing support for deterministic figure output #196

Closed astrofrog closed 1 year ago

astrofrog commented 1 year ago

This is a work in progress and not ready for review

astrofrog commented 1 year ago

This should be ready for review. One question is whether this should actually be the default, at least for the new supported formats. For PNG we would break people's tests if they have different baseline images for different matplotlib versions and were not setting the Software: None metadata. But for the new formats it might be sensible?

astrofrog commented 1 year ago

@ConorMacBride - thanks for reviewing! I'm a little uncertain how we should approach breaking changes at this point (in the sense that we would still break a lot of CI/testing in existing packages, so perhaps we need to batch open issues to repos using the plugin), but of the changes you mention I agree that if we could make breaking changes, then switching the default Matplotlib style would be good. Perhaps also changing the default tolerance to 0. However I'm not sure about making perceptual hashes the default, I still feel that should probably be opt-in? In any case perhaps we could open a new issue to discuss it?

astrofrog commented 1 year ago

Or at the very least we'll need to have deprecation warnings in place for a little while before we actually break things.