Get all languages working in prybar with Nix

Minivera commented 3 years ago

Some languages were not working in prybar for multiple reasons. This PR implements a good number of fixes to the nix files, the test files, and the main.go file for multiple languages. The result is the ability to run nix-build to build all the prybar languages and have them tested in the nix sandbox. They are all properly configured for the languages to run and for the prybar to access the helper files/binaries.

Some notes

I have added comments to the nix files, but here are a few things of note from this PR:

This might break the old tests since out tests have version numbers embedded into them.
Not all languages are on the latest version, but I made sure to get as close to latest as possible. My rule of thumb was to focus on using the "official" package from nix (for example, ruby rather than ruby_3_0) first and try to work around those.
The expect test files are now executed in the check phase of Nix so all build are now reproducible, even in their test phases. Nothing can be fetched from the internet since the builds run in a sandbox, meaning some work is needed to make sure a language has ALL dependencies.
Most of the nix code is generic, but that leads to some complicated code around the building phase. What's important to know is that we can pass buildInputs for the build phase and binaries for the test phase, neither of which will be added to the final package. We can then use the setFlags attribute to tell nix to add the path to the binaries' source for the go compiler. This was needed for all the languages using cgo. As long as no language throws a curveball, these 3 flags should make sure that all languages are buildable and testable.

List of language changes:

Prybars versions in Nix are now based on the version number from the main prybar nix file, they won't create their own version number and it can be passed as an argument.
Wrapped all the clojure deps in a clojure package that uses the official clojure binary. This stops nix from fetching the dependencies from the internet in the sandbox.
Added our own version of Julia that loads the 1.5.4 sources files without building. It also adds a pkg-config file to help make the code more generic. 1.6.1 does not work with the way prybar is built right now (They have scattered the C binaries)
Made all go:generate commands use bash as the script interpreter to standardize things and make sure they have access to the env.
Added some code to add all ocaml deps from the OCAMLPATH as arguments to the binary since Nix is too isolated to be able to find them.
Removed spidermonkey
Fixed some of the tests that were breaking due to new versions (Including, sadly, skipping the tcl 1+1 test due to some issues with how tcl returns expressions in the newer versions.

Testing

Test with nix-build, it should resolve after some time with all prybars working as intended and all tests passing. If it fails on your machine, then I made a mistake and things are not reproducible, somehow!

CI?

CI will be added in a follow-up PR, this might break out current CI implementation since the version numbers are hardcoded in some of the tests. I will see how I want to resolve this (probably by keeping two copies of the test files)

Why does it say revert in the commit history?

You saw nothing

cbrewster commented 3 years ago

What do you think about maintaining this in a separate branch for now until we are fully on Nix'd pid1 and not going back? Mainly so we can keep compat with the old ways until the old ways are done.

Minivera commented 3 years ago

No against it, from my tests, it should be fully backwards compatible except for the ruby version change. Will that version change impact the backwards compatibility of prybar in pid1?

cbrewster commented 3 years ago

Potentially, it may conflict with the version of Ruby we have in Polygott (Ruby 2.5).

cbrewster commented 3 years ago

Oh wait, I don't think we prybar Ruby. In that case, lets ship it to master!

Minivera commented 3 years ago

I've checked the files and the other two languages that could have backwards compatibility issues are R and spidermonkey, depending on if they're used at all.

cbrewster commented 3 years ago

Those aren't prybar'd yet either AFAICT

replbot commented 3 years ago

prebooping: tests are pending

Minivera commented 3 years ago

I don't have the full context on the impact of this on pid1, so I'll trust your judgement. I'm more than happy to move it to a draft and wait until pid1 is nix'ed if you feel that's the safer approach :slightly_smiling_face:

replbot commented 3 years ago

unbooping: approved

cbrewster commented 3 years ago

I don't have the full context on the impact of this on pid1, so I'll trust your judgement. I'm more than happy to move it to a draft and wait until pid1 is nix'ed if you feel that's the safer approach :slightly_smiling_face:

We aren't shipping this in pid1 yet (maybe ever) so not too worried. I think we may want to expose these in our overlay and make our overlay available by default inside a repl. The users can pull in a prybar if they'd like and we can provide templates with prybar already set up.

I kinda view prybar in a similar vein as LSP where in the future it should be plug and play and people can bring their own.

replit / prybar