geospace-code / georinex

Python RINEX 2 / 3 NAV / OBS / sp3 reader & batch convert to HDF5 with C-like speed
MIT License
216 stars 89 forks source link

Use Hatanaka library for crx2rnx decompression #73

Closed valgur closed 2 years ago

valgur commented 3 years ago

Hi!

GeoRinex already bundles and wraps the crx2rnx executable quite nicely, but I'm sure you agree it's less than ideal by being a pain point for Windows users and adding a dependency for a C compiler to be present.

I thought that such a basic requirement for working with RINEX data could use a more reliable and convenient solution, so I created a separate library called Hatanaka that focuses on just that.

It builds the executable during installation and makes use of the default C compiler detected by setuptools, making things much easier for Windows users. Even more importantly, it also pre-builds and packages the executable as a wheel for all major operating systems. This needed quite a bit of hacking of the setup script since wrapping a binary executable is clearly not something setuptools authors had planned for. Still, I managed to get it to work nicely and reliably for all package formats and operating systems, as can be seen by the extensive test suite and CI coverage.

It also makes use of some of the good ideas in this package, such as the suggestion to use importlib_resources and the automatic C compiler detection as a fallback in case Python fails to find one. So, thanks for that.

I also added support for more general decompression since you almost never encounter a RINEX file with only Hatanaka compression. I did not add that to the PR though, since GeoRinex already covers that well enough.

Two additional minor additions:

I hope you find it useful. Feel free to provide any feedback on the library itself and I'm more than happy to make any changes to the PR if you disagree with anything.

scivision commented 3 years ago

I think this can be done with the vendored crx2rnx or your hatanaka module. That is, if the user's computer isn't capable of compiling crx2rnx, I can make the error message suggest your package as an alternative. I can rework this to have the same API for the vendored and external packages.

A reason for minimizing even trivial pure-Python prereqs is to help allow this georinex to work easily on offline systems, where they probably already have the common numerical modules like Numpy.

scivision commented 2 years ago

this was implemented for https://github.com/geospace-code/georinex/releases/tag/v1.16.0