Relative paths when using `nix-shell` as an interpreter

[stolyaroleh]

Hi everyone!

At work, we have a huge monorepo and a frozen version of nixpkgs to build services and libraries. There's a release.nix file containing all our packages and it's very nice to know that if it can nix-build, then all the services compile and pass unit tests.

I've been trying to make our helper scripts reproducible by using nix-shell as a script interpreter. Unfortunately, there seems to be no way of pointing them to our frozen nixpkgs in a nice way. Given the following folder structure:

- nix
  - nixpkgs
    - default.nix # frozen nixpkgs
  - bin
    - helper-script.py

I can write a helper-script.py that looks like this:

#!/usr/bin/env nix-shell
#! nix-shell -I "nixpkgs=." -i python -p "python.withPackages (pkgs: [ pkgs.foo ])"
import foo
foo.blah()

...but it will only work if invoked from the nix directory as ./bin/helper-script.py. It would be really awesome if it was possible to specify pinned nixpkgs relative to the interpreted file if using nix-shell as an interpreter. In this way, no matter how we run that script, it would always resolve nixpkgs to what we have pinned right now.

I looked at the source code of nix-build (https://github.com/NixOS/nix/blob/master/src/nix-build/nix-build.cc#L111), and there seems no way to make it work now. Do you think adding a feature like that would be useful? How would it look?

...one idea I had is to introduce a new builtin - interpretedFile, that is only available when running nix-shell as interpreter. Since the value of <nixpkgs> is normally a path I want to import, I was wondering if I could make this work by putting a Nix expression in the -I:

#! nix-shell -I "nixpkgs=\"with builtins; (dirOf interpretedFile) + ../nixpkgs/\"" -i python -p "python.withPackages (pkgs: [ pkgs.foo ])"

[zimbatm]

see https://discourse.nixos.org/t/relative-paths-when-using-nix-shell-as-an-interpreter/1180

[tbenst]

I wrote a (hack) solution using a polyglot!

/tmp/test:

#!/usr/bin/env bash
"exec" "$(which nix-shell)" "$(dirname ${BASH_SOURCE[0]})/../shell.nix" "--run" "python $0 $*"

__doc__ = """module docstring"""
import sys
print(sys.argv)

> /tmp/test hello "world"
['/tmp/test', 'hello', 'world']

The better solution would be for Nix to have a interpretedFile as you suggest

Edit: note this does not support () without escaping for unknown reasons that seem to be nix-shell's fault. hopefully https://github.com/NixOS/nix/issues/534 or https://github.com/NixOS/nix/issues/1189 will be addressed and solve this issue

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/relative-paths-when-using-nix-shell-as-an-interpreter/1180/4

[xzfc]

@tbenst

Edit: note this does not support () without escaping for unknown reasons that seem to be nix-shell's fault.

It behaves same as bash (e.g. bash -c $0 "python $*"), you need to quote it.

#!/usr/bin/env bash
"true" '''\'
exec nix-shell "$(dirname ${BASH_SOURCE[0]})/../shell.nix" --run "$(printf '%q ' python "$0" "$@")"
'''

I'm also unsure why you use "$(which nix-shell)" instead of just "nix-shell".

[expipiplus1]

For any unfortunate souls stumbling on this here's the Haskell version, necessary because GHC doesn't find modules relative to a source file without -i/directory/of/source/file

#! /usr/bin/env bash
{- 2>/dev/null
# Use some stupid polyglot here so that we can add the directory of this script
# as an include path to GHC.
DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"
exec nix-shell \
  --pure \
  -p "haskellPackages.ghcWithPackages (hp: with hp; [ foo bar ])" \
  --run "runghc -i$(printf "%q " "$DIR" "$0" "$@")"
-}

module Foo where 
...

[stale[bot]]

I marked this as stale due to inactivity. → More info

[roberth]

For most arguments, this has been fixed in Nix 2.24, but I'm not sure about -I. Is this still a problem?