Publish on conda and nix

[alippai]

I'd be happy to use datafusion-cli without the need for the rust toolchain. For the end-users nix and conda (conda-forge) are more common package managers.

Describe alternatives you've considered Homebrew, winget might be early, they are less dev focused.

Additional context Are there special requirements by Apache? I assume it would need to be managed in a centralized way.

[cpcloud]

I've put up a conda package recipe for datafusion here https://github.com/conda-forge/staged-recipes/pull/17387.

I just built it locally (on Linux), and the build passes.

Packaging for nix is likely out of scope for datafusion maintainers, though it's not too bad to package up a Rust-backed Python package (see https://github.com/NixOS/nixpkgs/blob/a0dbe47318bbab7559ffbfa7c4872a517833409f/pkgs/development/python-modules/cryptography/default.nix#L24) in nixpkgs. Such a PR for datafusion would likely be merged shortly after a successful nixpkgs CI build.

[matthewmturner]

@alippai FYI i believe datafusion-cli is available on brew now (https://github.com/apache/arrow-datafusion/issues/562)

[cpcloud]

nixpkgs pr here https://github.com/NixOS/nixpkgs/pull/152763

[alippai]

@cpcloud really nice, also it looks like it uncovered an upstream issue with the lock file, right? This is for the Datafusion python module / bindings and the Datafusion CLI would be a separate module?!

[cpcloud]

@alippai Yeah, I believe datafusion-cli would be a separate conda and nix package

[cpcloud]

Regarding the lockfile, yeah python/Cargo.lock is out of date. I fixed that upstream here https://github.com/apache/arrow-datafusion/pull/1508, just waiting on a review from a maintainer.

[alippai]

Thanks for the details