search

aperiosoftware / astropy

Repository for the Astropy core package
www.astropy.org
BSD 3-Clause "New" or "Revised" License
0 stars 0 forks source link

Add quantization codec and implement compression/decompression of floating point data #31

Closed astrofrog closed 1 year ago

astrofrog commented 1 year ago

Fixes https://github.com/aperiosoftware/astropy/issues/14 Fixes https://github.com/aperiosoftware/astropy/issues/40

github-actions[bot] commented 1 year ago

👋 Thank you for your draft pull request! Do you know that you can use [ci skip] or [skip ci] in your commit messages to skip running continuous integration tests until you are ready?

astrofrog commented 1 year ago

I think lossy tests are close to passing but due to the issue mentioned in https://github.com/aperiosoftware/astropy/issues/40 the floating point values don't match exactly:

E       Mismatched elements: 144 / 144 (100%)
E       Max absolute difference: 7.677353
E       Max relative difference: 7.3442245
E        x: array([[163.9622  ,  12.696566,   2.805807,  83.07929 ,   3.645739,
E                12.367053, 118.553955,  33.24904 ,  59.632137, 209.43936 ,
E                23.05459 ,  67.77326 ],...
E        y: array([[1.626797e+02, 9.415493e+00, 1.536737e+00, 8.256937e+01,
E               5.992008e+00, 9.394189e+00, 1.165274e+02, 3.576727e+01,
E               6.091895e+01, 2.120618e+02, 2.193155e+01, 6.678799e+01],...

I think we should adjust the tests to compare the data read in by both libraries rather than comparing to the very original data.

astrofrog commented 1 year ago

@Cadair - I found some time to implement the comments