Support opening arbitrary FDs for child processes

[qwandor]

Duct provides methods like Expression::stdin_file and Expression::stdout_file to use some existing file for stdin and stdout (i.e. file descriptors 0 and 1), but there doesn't seem to be any way to give a child process an arbitrary set of file descriptors. It would be useful to be able to run a child process with an arbitrary set of file descriptors, like you can from the shell e.g.

./command 5<somefile.txt 6>someotherfile.txt 7<>athirdfile.txt 8<&4

This is useful because it allows the parent process to give the child access to a file that the child doesn't have permission to open itself.

[oconnor663]

Currently you can do this by going through std::fs::File::from_raw_fd. However, this gives ownership of the descriptor to the File and to the Expression, and it'll be closed when the Expression is finally dropped. If you want to keep the descriptor for yourself, you could try to abuse ManuallyDrop together with File::try_clone, or you could reach for something like libc::dup2 directly.

How does that sound? Does it fit your use case?

[qwandor]

I'm not quite sure how from_raw_fd helps. I want to be able to run some other program as a subprocess, starting it with an arbitrary set of file descriptors. For example, suppose I want to open a file and pass it to the subprocess as stdin (i.e. FD 0), I can do:

let file = File::open("file.txt")?;
cmd!("subprocess").stdin_file(file).run()?;

I'd like to instead be able to pass it as some other FDs, e.g. 5 and 6, something like:

let file = File::open("file.txt")?;
let file2 = File::open("file2.txt")?;
cmd!("subprocess").fd_file(5, file).fd_file(6, file2).run()?;

(But in reality, the File would be passed in from somewhere else, not something I just opened for the purpose.)

This would indeed need to call dup2, but it's a bit more complicated than that because FD 5 might already be used in the parent process for some other file. So the dup2 call needs to happen after the fork and before the exec, so that it only affects the child process.

[oconnor663]

Oh I misunderstood two things. I thought you wanted to use arbitrary file descriptors from the parent to set the standard descriptors in the child, but that's backwards. Now I see you actually want to set nonstandard descriptors in the child. Also I got confused between dup2 and pipe2, and what I meant was dup. (But now I understand that dup2 is indeed going to be involved one way or another.)

To get something like this done with Duct, you'll need to go through duct::Expression::before_spawn and from there through std::process::Command::pre_exec, so that you can make all the necessary libc calls in that pre-exec context. As you probably know, that context tends to be pretty restrictive about which functions you're allowed to call, which is why pre_exec itself is unsafe. You'll need to do your dup2 dance there (if you're aiming for fd 3, make sure none of the other files you're planning on using is already fd 3, etc.), and depending on where the fds come from you might need to unset the CLOEXEC flag.

In general, Duct considers this sort of use case to be a bit beyond its target. It's pretty unusual to want the level of fine-grained control you want here, but also to want stuff like automagical stdout capture. Also these things are platform specific, and not generally portable to Windows. Have you considered using std::process::Command directly?

[qwandor]

Aha, I didn't see before_spawn. Yes, using std::process::Command directly (with shared_child) is what I'm currently doing, but I was hoping to find something with a nicer API.

If this is out of scope for Duct then feel free to close this feature request, and I'll make my own crate (unless you know any other wrapper crates that would make sense for this to be part of).

[oconnor663]

Yeah I think I'm going to close this one as "too advanced/specific" for Duct, but it's possible that https://github.com/hniksic/rust-subprocess might be interested in such a feature? Alternatively, having this in a standalone crate might be valuable, especially if its only dependency was std::process::Command (and presumably libc). Even having a centralized document covering "what edge cases do you need to worry about when doing this, what platforms does it work on" would be neat.