Closed oxinabox closed 3 years ago
For 1.0 the problem is that it can't resolve it's test-time dependencies. Because 1.0 won't reresolve dependencies at test time. See https://www.oxinabox.net/2021/02/13/Julia-1.6-what-has-changed-since-1.0.html#resolver-willing-to-downgrade-packages-to-install-new-ones-tiered-resolution
For other things, It seems to be because the plot is slightly different. so I guess need to adjust the sensitivity or find a way to make the plot less sensitive to changes.
Failure on 1.0 Ubuntu. idk why. They are indeed slightly different
Thank you @oxinabox for pushing this forward ❤️
@juliohm do you have any ability to dig into what is going on with the heatmap on ubuntu?
I am on Manjaro Linux so maybe the issue would reproduce there as well? How can I help? I understand very little about ImageIO and about these possible discrepancies between OSes.
I am on Manjaro Linux so maybe the issue would reproduce there as well? How can I help? I understand very little about ImageIO and about these possible discrepancies between OSes.
Can you check out this branch and see if tests pass for you locally?
I get a couple of warnings but all the tests pass.
hmmm, Not sure what to do here. Maybe mark it conditionally broken and open an issue? Maybe we can up the tolerance? idk how to up tolerence
I had the same question in the packages where I am using ReferenceTests.jl How to increase tolerance? The current threshold with PSNR is not very intuitive, at least for me, and guidelines are appreciated. Perhaps increase the default tolerance in ReferenceTests.jl would also be a good idea.
Any thoughts on that @johnnychen94 ? It would be nice to fix this OS-specific issue, perhaps just increasing tolerance for now.
Warning: test fails because PSNR -22.433345794677734 < 15
for the heat map
OK, I have spun up a linux VM and replicated the failure, not that i can replace it, fixing it should be easier
Confirmed it is Plots/GR actually making a clearly different output on linux.
This is https://github.com/JuliaPlots/Plots.jl/issues/2127 Now that i know it is a know bug, I feel ok just disabling this test til it is fixed. It's not our problem to fix.
If there are no objections I will merge this tomorrow
Including plots.
Closes #62 Alternative to #76
cc @juliohm