search

approvals / ApprovalTests.LabVIEW

MIT License
1 stars 2 forks source link

Marginal differences shouldn't fail the test (there should be an option for this - sometimes a marginal failure is important) #4

Open phorn-NI opened 1 month ago

phorn-NI commented 1 month ago

Caraya supports a margin of error when comparing floating point numbers (as 10.0 != 10.0 exactly)

I found a similar issue when generating png images and trying to compare them. The two attached images were generated by exactly the same configuration script and look identical, but not when you put them through a compare tool - however for my purposes they're close enough (its the ends of the error bars that are 1 or 2 pixels different).

Its not urgent as the work around is to compare the configuration script, which is generated from a list of parameters - and that's what I'm really trying test Test error-bar override test error bar base json formatter approved Test error-bar override test error bar base json formatter received

stagg54 commented 1 month ago

Hmm... I imagine this is fixable, but it will require some research. There is probably a png compare tool out there that might work for something like this. Right now IIRC (without looking at the code) I'm pretty sure it's just checking the file hashes to see if they match. That would have to tolerance builtin. That comparison is overrideable though - for VI and CTL files, it uses the LVCompare function.

phorn-NI commented 1 month ago

Interesting research project then 😄 As I said, it's not a big problem for my immediate use case - testing the generated json is robust and gives me the same info.

stagg54 commented 1 month ago

Yeah I did a very brief look. ImageMagick seems to have a compare tool. Not sure how well that would work.