search

JuliaTesting / ReferenceTests.jl

Utility package for comparing data against reference files
https://juliatesting.github.io/ReferenceTests.jl/latest/
Other
82 stars 14 forks source link

When displaying failure link to files to open externally? #79

Closed oxinabox closed 3 years ago

oxinabox commented 3 years ago

When we display a failure, it could be useful to display not just our rendered difference, but also display a link that the user can click to open each in an external tool.

This is particularly relevent for images where we use ImageInTerminal to display at low resolution. But if the difference is subtle that won't show up.

To do this for the new content we would need to write it to a tempfile, and display a link to that.

oxinabox commented 3 years ago

A cool thing we could do with this. Rather than just always save to a tempdir, we could have it be configurable via an enviroment variable (default to mktempdir)

Then during CI we could configure it to upload that directory at the end, as an artifact. Which would make it easier to debug things failing on CI https://docs.github.com/en/actions/guides/storing-workflow-data-as-artifacts