search

dirkwhoffmann / vAmiga

vAmiga is a user-friendly Amiga 500, 1000, 2000 emulator for macOS
https://dirkwhoffmann.github.io/vAmiga
Other
293 stars 24 forks source link

/tmp no longer accessible for running regression tests #804

Closed dirkwhoffmann closed 6 months ago

dirkwhoffmann commented 6 months ago

In macOS Sonoma, it seems like Apple restricts access to the /tmp folder in a way that breaks vAmiga's regression testing system. E.g., it's no longer possible to copy the vAmiga executable to /tmp and run it directly from there within a shell.

TODO: Make the regression testing framework more flexible. I.e., make it work to run inside a local folder.

dirkwhoffmann commented 6 months ago

Update: Everything seems to work fine if the vAmiga executable is loaded from somewhere else than tmp. However, all regression tests now fail because tiffcmp reports a difference: 

Scanline 38, pixel 164, sample 0: f0 ef

I don't see a difference:

pic1

pic2

To make a long story short: I had trouble with comparing TIFFs multiple times in the past, for whatever reason. Comparing TIFFs basically breaks from each OS release to the next.

TODO: Let the regression tester compare the raw image data. Save TIFFs only for visual control of the result.

dirkwhoffmann commented 6 months ago

The regression testing framework has been updated. Reference images are now stored and compared in raw format. Hopefully, this solution will prove more stable over time than the existing one.