search

go-p5 / p5

p5 is a simple package that provides primitives resembling the ones exposed by p5js.org
https://go-p5.github.io/
BSD 3-Clause "New" or "Revised" License
146 stars 12 forks source link

Tests are fragile due to image comparison #17

Closed psampaz closed 3 years ago

psampaz commented 3 years ago

I am trying to run locally the test suite but it fails due to existing delta of 0.05

=== RUN TestAPIShapes api_test.go:61: testdata/api_shapes.png: images compare different --- FAIL: TestAPIShapes (0.08s)

Increasing delta to 0.06 make test green.

Is there any other more robust way to implement tests avoiding image comparison completely? Is there a way to compare the state that produces the images instead of the images themselves?

sbinet commented 3 years ago

I don't know of another, better, way.

the magical value of 0.5 is just that: a magical value, which can be increased by experimentation.

that said, it's possible the discrepancy is coming from the Gio version being used. I haven't updated Gio (nor the reference files) in a somewhat long time and Gio has seen a few updates in the path/strokes department.

let me update the Gio stack.

sbinet commented 3 years ago

see https://github.com/go-p5/p5/pull/18. do the reference files work better for you?

(what platform/os are you working with, btw?)

sbinet commented 3 years ago

BTW, I've created this little program to see differences (and the min/max of those) between 2 images:

psampaz commented 3 years ago

now it is fine, thanks 👍