Allow offering dynamic script output for download

[soenkehahn]

@casey: I was wrong about how processes get executed on linux: The kernel is capable of running scripts and binaries (e.g. ELF binaries), but in the case of scripts, they have to start with #!. So the kernel will not fall back on any shell, if the hashbang is missing. You can run scripts that don't start with #! from e.g. bash, and that will work by falling back on some shell, but that behavior is implemented in the shells you're using for that. (And apparently the exact behavior (e.g. which shell to use) depends on the parent shell being used.)

So this PR right now implements the following logic: When the given script starts with #! it is executed unmodified. If it doesn't a hashbang line is added (#!/usr/bin/sh). Judging from the tests that seems to be the behavior we want, but it's a bit magical.

The PR still needs to be cleaned up a lot, but I wanted to get your feedback early.

[casey]

@casey: I was wrong about how processes get executed on linux: The kernel is capable of running scripts and binaries (e.g. ELF binaries), but in the case of scripts, they have to start with #!. So the kernel will not fall back on any shell, if the hashbang is missing. You can run scripts that don't start with #! from e.g. bash, and that will work by falling back on some shell, but that behavior is implemented in the shells you're using for that. (And apparently the exact behavior (e.g. which shell to use) depends on the parent shell being used.)

I think I'm still a little confused about what the behavior is, and where the inconsistency is from. What you're saying is when you do ./foo in bash and foo is a text file that doesn't start with a #!, then it's bash that chooses the shell to run it with, not the kernel? What happens if you try to run it with std::process::Command::new("./script")? (I.e. with no intermediate shell.)

On macOS, running a text file with no shebang with std::process::Command::new seems to work. I think it uses /bin/sh, but I'm not sure how it makes that decision. (/bin/sh on macOS is bash.)

I think adding a shebang automatically might not be desirable. It would make it unclear what was running a script without reading documentation, and it would mess up line numbers in error messages. Also if a user misspelled a shebang, i.e. added !#/usr/bin/env, we would add our own, which would be confusing.

I think it would probably be okay to just chmod +x the script and run it, and let the user add the #! if their system requires it.

[soenkehahn]

I think I'm still a little confused about what the behavior is, and where the inconsistency is from. What you're saying is when you do ./foo in bash and foo is a text file that doesn't start with a #!, then it's bash that chooses the shell to run it with, not the kernel? What happens if you try to run it with std::process::Command::new("./script")? (I.e. with no intermediate shell.)

I get:

[src/main.rs:2] std::process::Command::new("./foo").output() = Err(
    Os {
        code: 8,
        kind: Uncategorized,
        message: "Exec format error",
    },
)

On macOS, running a text file with no shebang with std::process::Command::new seems to work. I think it uses /bin/sh, but I'm not sure how it makes that decision. (/bin/sh on macOS is bash.)

Weird.

I think adding a shebang automatically might not be desirable. It would make it unclear what was running a script without reading documentation, and it would mess up line numbers in error messages. Also if a user misspelled a shebang, i.e. added !#/usr/bin/env, we would add our own, which would be confusing.

I think it would probably be okay to just chmod +x the script and run it, and let the user add the #! if their system requires it.

Yeah, good points. I think that sounds right.