This PR is a second iteration on Dataframes building & packaging automation. It adds building relocatable Linux (x64) packages that are uploaded to S3 (just like previously we had done with Windows packages).
The packaging script has been developed into full-fledged Haskell package. While the current stage is still largely provisional, I attempted to keep code organized and documented.
The code dealing with calls to external programs has been reorganized — each program got its own module. I have added a typeclass to provide uniform interface and sensible run-time diagnostics without code duplication. Not sure if this is idiomatic Haskell but I haven't got any better ideas.
The code intentionally doesn't attempt to deal with any other OS than Windows and Linux. It is expected that Mac support will be added in the next iteration.
As for the Linux packaging itself, it is slightly different from Windows:
dependencies are provided by the Docker image we use
Docker image is based on Centos 7 so binaries are glibc-wise portable enough
build is CMake/GCC-7 based
packaging is driven by ldd and rpath patching
whole thing has been added as a new job in CircleCI
Python package skeleton is just built from sources Python installation, likely could be streamlined more (though there is no out-of-box embaddable build option like on Windows)
This PR is a second iteration on Dataframes building & packaging automation. It adds building relocatable Linux (x64) packages that are uploaded to S3 (just like previously we had done with Windows packages).
The packaging script has been developed into full-fledged Haskell package. While the current stage is still largely provisional, I attempted to keep code organized and documented.
The code dealing with calls to external programs has been reorganized — each program got its own module. I have added a typeclass to provide uniform interface and sensible run-time diagnostics without code duplication. Not sure if this is idiomatic Haskell but I haven't got any better ideas.
The code intentionally doesn't attempt to deal with any other OS than Windows and Linux. It is expected that Mac support will be added in the next iteration.
As for the Linux packaging itself, it is slightly different from Windows: