Closed tomeichlersmith closed 4 months ago
The shebang #!
syntax is not very flexible and breaks portability of a lot of scripts since it is handled by the kernel [^1]. Kernels of various types and versions change how long this shebang line can be (in terms of characters) sometimes going as low as 16 characters (really old). The more common limit is a few hundred characters which can quickly be reached by a longer path to the denv_workspace
.^2 Some solutions exist that I will look at adapting to our case. https://github.com/spack/sbang is of particular interest since I could foresee denv
acting like sbang
does.
[^1]: See this unix stackexchange answer: https://unix.stackexchange.com/a/29620
Getting the shebang working with an already-accessible workspace was rather simple. Drawing heavily from sbang as mention above https://github.com/tomeichlersmith/denv/issues/92#issuecomment-1992182401, I was able to parse the first few lines of the file and set denv_workspace
to the value defined in those lines. This enables scripts like (assuming #
is the comment character for the command)
#!/usr/bin/env -S denv shebang
#!denv_workspace=/full/path/to/workspace
#!/command/to/run/in/denv
code for command in denv
The command in the denv is not passed to the kernel like a shebang, so it doesn't need to be a fullpath. It is passed to sh -lc
like any other instance of running denv cmd
which makes it a bit more flexible than a traditional shebang.
/usr/bin/env -S denv shebang
or installing a separate script named denv-shebang
or something (to avoid requiring the -S
being passed to env
.denv_workspace
is defined outside the script. We don't want folks to define it as a part of their host environment because that defeats the purpose of having a denv.I am interested in making the denv shebang something that could fully specify the denv such that a workspace does not need to be specified. This would make denv more usable in a cluster context where the workspace may not be shared amongst worker nodes and it could be useful for sharing work since the command would run the same as long as the users had denv installed. Supporting this running mode in a stable way would introduce some complexity.
.denv/config
or it could be loaded from the shebang lines (perhaps with some defaults that we'd want to share with denv init
as well). This would mean I'd probably want some sort of "config validation" so that we could check that both configs are valid.These two roadblocks are causing me to pause and evaluate what I want to do. Perhaps, I will merge my workspace-required version soon (after those To Dos) and start a separate issue for workspace-less version.
Is your feature request related to a problem? Please describe. I'd like to write scripts that run programs in a denv. The easiest approach to do this is to write a normal script and then execute it with
denv
:But sometimes, this execution path is unavailable or bloated. For example, sometimes I want to run
my-script.sh
in a denv that resides somewhere else. The currently supported way to do this isIt is a hassle to type out the full denv_workspace path and sometimes its not possible to do (for example in batch processing contexts). This forces me to write a second script that can wrap
my-script.sh
which unfortunately means another round of shell interpretation when expanding
$@
and another file to carry around.Describe the solution you'd like It'd be really cool if I could specify
denv
with a shebang so thatmy-script.sh
is automatically run within the denv. Something likeThis would allow me to have a single file whose first line specifies that it is run by
denv
I've tried this, but it does not work out of the box. It appears to hang probably due to my misunderstanding how the script is given todenv
when it is specified in the shebang.This solution could be expanded by adding a
denv
option specifying what should be run within the denv.Or using the current "remote" running capability
Describe alternatives you've considered A wrapper script like shown above may be able to function as a shebang, but it still would introduce the bloat of an additional script to carry around.
Additional context GNU parallel has a
--shebang
option: https://www.gnu.org/software/parallel/parallel_tutorial.html#shebangLooking at the source
parallel
and searching for--shebang
reveals that it needs to re-execute itself when acting as a shebang. It looks like we want to mimic--shebang-wrap
so that the user can telldenv
which program to give the script to within the container.