support nix package

[bendlas]

I've packaged pixie with nix: https://github.com/bendlas/nixpkgs/tree/a15e55bdbe956273f0be3915f8c325a958080fb2/pkgs/development/interpreters/pixie (the .patch file, in particular)

I think that nixos can offer tremendous value to pixie:

• binary distribution (nix can work within another distro, like npm)
• stable dependencies
• ease of configuring vms, docker images, ...

In order to get pixie built, I had to do some work, that I'd like to offer back upstream. This will also benefit other distributions, that would like to package pixie:

• [ ] I basically recreated the whole makefile as a nix expression. I had to do this, because the included makefile seems pretty ad-hoc, downloading unversioned dependecies, creating stuff in the parent directory. I'd love to see a more packaging-friendly makefile, so that I can reduce the complexity of the build expression. But more importantly, I had to patch pixie's source for two issues and we should focus on getting them patched first:
• [ ] pixie hard-codes pathes relative to its executable, as load-paths. Its load-paths are also the only place it looks for c header-files and library objects. My current patch replaces those relative paths with stable (crypto-hash) pathes to the c-dependencies it mentions in the docs (+ zlib, which it doesn't document). A proper solution should have a configuration phase, where it spits the pathes from pkg-config into some sort of config.edn
• [x] pixie's ffi generates a file /tmp/tmp.cpp and compiles it. This is not concurrency-safe, as well as a security hole + it doesn't work when /tmp is not writable (as is the case within a nix-build). My patch replaces this with a usage of (getenv "TMPDIR") + mkdtemp to create a temporary directory to compile its ffi info in.

It should be clear, that all of this are issues, not just concerning nixos, but any distribution, really. However, I didn't create a pull request right away, because I'd like to get feedback from pixie's maintainers, before reworking its build process. If you are interested and also willing to keep half an eye on integration, I'd be willing push some patches to you, as well as get pixie into the nixos distribution.

[bendlas]

If you'd like to build if for youself, make sure you've installed the nix package manager, and then:

git clone https://github.com/bendlas/nixpkgs.git
cd nixpkgs
git checkout build-pixie
nix-build . -A pixie

[heyLu]

Thanks for doing this!

Can you create a pull request for the ffi tmpfile patch?

The load path issue is more complicated... I don't really like having to hard code these paths, I posted some thoughts on this in #429.

[bendlas]

I agree, the load-path issue is a bit tricky. From a distro-packager's point of view, pixie's binary/c path currently consists of a combination of:

• $LD_LIBRARY_PATH used by pypy's jit_libffi pixie/vm/libs/ffi.py
• pixie.stdlib/load-path used by pixie/ffi-infer.pxi, which is
  • initialized to . in target.py
  • augmented with lib, include, ../lib, ../include during initial loading. I couldn't easily move those to target.py either
  • augmented from cmd-line params after initialisation

What I think should happen:

• Only hardcode the path for pkgconfig in an easily configurable location
• Use pkgconfig to find include paths and library paths for the c compiler in ffi-infer
• Define a mapping to let the user work with libraries not known to pkgconfig. Reusing pixie's load-paths might be sufficient, if a bit awkward to not distinguish include-paths and lib-paths for c++
• Define loading precedence

This would be sufficient to cleanly package pixie in distros. Additionally, we might think about:

• Accept PIXIEPATH to initialize load-paths?
• Keep loading .so's from LD_LIBRARY_PATH, PIXIEPATH, both?

[bendlas]

Quick update: pixie and dust have landed in nixos-unstable. Therefore, I'm proud to say, nixos is the first distribution to ship binary packages for pixie and dust.

You can try it on your distribution by installing the nix package manager as per http://nixos.org/nix/ curl https://nixos.org/nix/install | sh this will set up nix on your system under /nix you can uninstall it by removing that directory. After installing nix, you can install pixie and dust by doing nix-env -iA nixpkgs.pixie and nix-env -iA nixpkgs.dust, respectively. You are very welcome to contribute to the pixie build expressions at https://github.com/NixOS/nixpkgs/tree/master/pkgs/development/interpreters/pixie

[vyp]

Sorry for off topic, but just for reference and convenience for future readers, the pixie executable via the above installation is named pxi.

[bendlas]

@vyp thanks for trying out the package. Should we rename pxi to pixie-vm? There wasn't really a standard, since we were the first to package it and I thought calling it pxi was kind of neatly conforming to the file extension. I understand though, that it's kind of hard to find when working by example ... Any thoughts, @heyLu

[heyLu]

I can only say I'd call the package pixie-lang or pixie, and the executable pixie.

[bendlas]

I feel, that it's a bit unconvential for linux packages to add the -lang suffix, not unheard of, though. If you still want it, I can push that package rename upstream.

Regarding the executable name, I think I'll just mirror the mainline build script, so, if somebody would update the Makefile + Examples to call #!/usr/bin/env pixie, I'll change the nix package accordingly.

Anybody else we should ask for blessing the package layout? Timothy did come up with much of the impl. Do you have any strong feelings on this @halgari?

[bendlas]

I've added a commit with pixie-lang and /bin/pixie-vm + deprecated /bin/pxi to my dev branch. Will push upstream, when consensus emerges.

Btw, I'm still in favor of /bin/pxi, because I feel it's a play on the lightweight nature of pixie, as well as on homoiconicity.

[bendlas]

Btw2, I'm sorry if my packaging job came off as trying to force a convention upstream. /bin/pxi was still in from my private packaging, while I kept the unwrapped, original executable in /share/pixie-vm, to be called by /bin/pxi. I didn't think of example code, spelling #!/usr/bin/env pixie-vm, sorry for that. That said, if we were to rename the executable, still consider /bin/pxi, due to arguments from previous comment.

[vyp]

@bendlas No, first of all, thank you for packaging it and spending all the time making the nix expression and this detailed ticket!

Original intent of my comment was to just do some extra documentation so future readers/users wouldn't give up or think something went wrong, and/or to save time because it took me a few minutes to find the pxi binary name in the nix expression.

But if you want my input on naming (for whatever it's worth), I'd have to vote against pixie-lang for the package name, just because it's unusual as @bendlas already said. I'd simply go for pixie as it already is (in nix). Only reason for pixie-lang would be if something else by the name of pixie already exists? (Say in Debian or something.)

For binary name I'd also just go with pixie, just because it's similar to interpreters for other languages, e.g. python, ruby, perl (instead of them using py, rb, pl). But of course, these names aren't always the same in every case (e.g. haskell -> runhaskell), so it's not really a big deal imo. But maybe you guys don't want to be boring! And so I agree too with pxi being a play on the lightweight nature...!

But pxi or pixie or pixie-vm for the binary name, @bendlas I'd just like it to be the same as whatever the Pixie project chooses (in Nix). It would just makes things less confusing and easier. Of course, if we have a different personal preference, we can always just use a custom local nix expression for ourselves to override the official nix expression in nixpkgs right? (Or simply a shell alias if it's a matter of typing pxi vs pixie.) (I guess you already knew that though, and you meant this more as choosing a default for everyone.)

[heyLu]

(I'd also prefer to have the package called just pixie, I only suggested pixie-lang because of possible conflicts. If there are none, pixie is a good name! :))

[alekcz]

I'd vote for pixie. I think is more user friendly towards new users having the binary named pixie (from personal experience). "To run pixie, I type pixie" In the mean time, I'm going to work on changing dust to work with pixie-vm,pixie and pxi.

[bendlas]

After seeing a couple of years without picking up significant usage, pixie is on track to be removed again from NixOS.

I believe there is still substantial value for the pixie project to be had from maintaining a Nix build descriptor. Luckily that needs not exist within the main distribution.

Either pixie could pick it up for doing CI and other builds, or otherwise I may move it to Nix User Repository

[alekcz]

@bendlas I think it's fair to remove it. The niche it filled has been filled and then some by babashka which is already on Nix. And the main contributors of pixie have moved on.