Comments on ProcessesCowan

[dpk]

• process-exec is portable to Windows, though process-fork is not. exec without fork is useful for hot-reloading a running program, among other things. I agree with removing the latter if we’re going to put this in the R7RS Environments.

• process-terminate-signal should be called something else so as not to be confused with something that has to do (only) with SIGTERM. (Cutesy, not entirely serious suggestion: process-cause-of-death.)

• I would make process-spawn take a fixed argument list like posix_spawn rather than an ersatz keyword argument structure.

• open-fds cannot be implemented portably on POSIX. You can get close, but not all the way. (posix_spawn is so close to being the Right Thing, spoiled by the fact they decided child processes should inherit all fds from their parents by default instead of the parent getting to set up the file descriptor table for its child afresh.)

• Is there any OS where stdin, stdout, and stderr are not 0, 1, and 2 respectively? I think this and the other file descriptor interfaces should be replaced by some kind of specialized file descriptor table type passed as an argument.

• There’s no way to set a timeout on process-wait. This might be the right thing (lower-case deliberate) since implementing this properly involves the self-pipe trick – but without a select interface you can’t do that yourself so we should provide for it.

[johnwcowan]

On Thu, Sep 5, 2024, 6:19 PM Daphne Preston-Kendal @.***> wrote:

-

process-exec is portable to Windows, though process-fork is not. exec without fork is useful for hot-reloading a running program, among other things. I agree with removing the latter if we’re going to put this in the R7RS Environments.

There is something on Windows called exec, but it's not equivalent to the Posix syscall family. However, the NT Executive, which líves below both Win32 and Posix, exposes Posix-compatible fork and exec as Nt* functions.

-

process-terminate-signal should be called something else so as not to be confused with something that has to do (only) with SIGTERM. (Cutesy, not entirely serious suggestion: process-cause-of-death.)

Sounds good to me!

-

I would make process-spawn take a fixed argument list like posix_spawn rather than an ersatz keyword argument structure.

Reasonable.

open-fds cannot be implemented portably on POSIX. You can get close, but not all the way. (posix_spawn is so close to being the Right Thing, spoiled by the fact they decided child processes should inherit all fds from their parents by default instead of the parent getting to set up the file descriptor table for its child afresh.)

-

Is there any OS where stdin, stdout, and stderr are not 0, 1, and 2 respectively?

Not that I know of.

-

I think this and the other file descriptor interfaces should be replaced by some kind of specialized file descriptor table type passed as an argument.

I don't and understand your reasoning here.

-

There’s no way to set a timeout on process-wait. This might be the right thing (lower-case deliberate) since implementing this properly involves the self-pipe trick – but without a select interface you can’t do that yourself so we should provide for it.

Fair enough.

[dpk]

On Thu, Sep 5, 2024, 6:19 PM Daphne Preston-Kendal @.***> wrote:

• process-exec is portable to Windows, though process-fork is not. exec without fork is useful for hot-reloading a running program, among other things. I agree with removing the latter if we’re going to put this in the R7RS Environments.

There is something on Windows called exec, but it's not equivalent to the Posix syscall family. However, the NT Executive, which líves below both Win32 and Posix, exposes Posix-compatible fork and exec as Nt* functions.

Oh, huh. I assumed since I knew MS-DOS had some kind of chain loading (and probably also because I found exec in section 3 of my manual, not section 2) that Windows must have some way to do it too.

I guess you can’t access the NT executive directly from normal user-mode program code?

• open-fds cannot be implemented portably on POSIX. You can get close, but not all the way. (posix_spawn is so close to being the Right Thing, spoiled by the fact they decided child processes should inherit all fds from their parents by default instead of the parent getting to set up the file descriptor table for its child afresh.)
• Is there any OS where stdin, stdout, and stderr are not 0, 1, and 2 respectively?

Not that I know of.

• I think this and the other file descriptor interfaces should be replaced by some kind of specialized file descriptor table type passed as an argument. I don't and understand your reasoning here.

Well, I guess my reasoning depends on whether we want the processes interface to directly reflect the POSIX Worse is Better attitude, or try to do the Right Thing.

As I say above, I think the Right Thing is that the parent specifies a table of file descriptors for the child when it starts the process. In other words, instead of the file_actions mess, posix_spawn should have taken an array mapping file descriptor numbers in the parent to file descriptor numbers for equivalent files in the child. The child doesn’t automatically inherit any fds; it gets only the ones you explicitly pass it. That is what I think the process library should also do if we want it to do the Right Thing, and we should create a special type which contains this file descriptor mapping, passed to process-spawn.

Unfortunately, the Right Thing is not what POSIX gave us. posix_spawn instead repeats the exact same mess that fork+exec has, in terms of controlling what open files a child process has access to based on what gets inherited and what you can dup2. You can use a number of tricks to get close:

• Just make sure all files you open have FD_CLOEXEC set. This is fine for many programs but as a new process you don’t know what file descriptors you have inherited, so any children you create will get those as well.
• Use the non-portable posix_spawn_file_actions_addclosefrom_np function. On platforms that support this, you can actually fully simulate the Right Thing. Unfortunately, it’s not portable for reasons I do not understand. The BSDs and apparently Solaris have it, Linux (well, glibc) and Mac OS do not. (The usual objections to closefrom as a general system call do not apply to its use as a file action.)
• Use a pile of hacks to try to simulate posix_spawn_file_actions_addclosefrom_np on the systems that don’t have it.

So, do we try to do the Right Thing by making implementers implement 90% solutions, or do we just accept that Worse is Better? If we think Worse is Better is the way to go, then we should probably wrap the file actions API more directly, as in sham1’s proposal

[johnwcowan]

On Tue, Nov 12, 2024, 10:59 AM Daphne Preston-Kendal < @.***> wrote:

On Thu, Sep 5, 2024, 6:19 PM Daphne Preston-Kendal @.***> wrote:

• process-exec is portable to Windows, though process-fork is not. exec without fork is useful for hot-reloading a running program, among other things. I agree with removing the latter if we’re going to put this in the R7RS Environments.

There is something on Windows called exec, but it's not equivalent to the Posix syscall family. However, the NT Executive, which líves below both Win32 and Posix, exposes Posix-compatible fork and exec as Nt* functions.

Oh, huh. I assumed since I knew MS-DOS had some kind of chain loading (and probably also because I found exec in section 3 of my manual, not section 2) that Windows must have some way to do it too.

I guess you can’t access the NT executive directly from normal user-mode program code?

No, you definitely can as of Windows 10.

• open-fds cannot be implemented portably on POSIX. You can get close, but not all the way. (posix_spawn is so close to being the Right Thing, spoiled by the fact they decided child processes should inherit all fds from their parents by default instead of the parent getting to set up the file descriptor table for its child afresh.)
• Is there any OS where stdin, stdout, and stderr are not 0, 1, and 2 respectively?

Not that I know of.

• I think this and the other file descriptor interfaces should be replaced by some kind of specialized file descriptor table type passed as an argument. I don't and understand your reasoning here.

Well, I guess my reasoning depends on whether we want the processes interface to directly reflect the POSIX Worse is Better attitude, or try to do the Right Thing.

I think we should adopt WiB except where it makes no sense, as when a signal interrupts a syscall that can logically be restarted, where tRT is appropriate. Another possible case is using a bytevector rather than a bv/offset/length triplet as a buffer. This is because of the large amount of existing documentation for the C API and the close resemblance of other language APIs.

As I say above, I think the Right Thing is that the parent specifies a table of file descriptors for the child when it starts the process. In other words, instead of the file_actions mess, posix_spawn should have taken an array mapping file descriptor numbers in the parent to file descriptor numbers for equivalent files in the child. The child doesn’t automatically inherit any fds; it gets only the ones you explicitly pass it. That is what I think the process library should also do if we want it to do the Right Thing, and we should create a special type which contains this file descriptor mapping, passed to process-spawn.

Unfortunately, the Right Thing is not what POSIX gave us. posix_spawn instead repeats the exact same mess that fork+exec has, in terms of controlling what open files a child process has access to based on what gets inherited and what you can dup2. You can use a number of tricks to get close:

• Just make sure all files you open have FD_CLOEXEC set. This is fine for many programs but as a new process you don’t know what file descriptors you have inherited, so any children you create will get those as well.
• Use the non-portable posix_spawn_file_actions_addclosefrom_np function. On platforms that support this, you can actually fully simulate the Right Thing. Unfortunately, it’s not portable for reasons I do not understand. The BSDs and apparently Solaris have it, Linux (well, glibc) and Mac OS do not. (The usual objections to closefrom as a general system call do not apply to its use as a file action.)
• Use a pile of hacks https://git.savannah.gnu.org/cgit/guile.git/tree/libguile/posix.c#n1338 to try to simulate posix_spawn_file_actions_addclosefrom_np on the systems that don’t have it.

So, do we try to do the Right Thing by making implementers implement 90% solutions, or do we just accept that Worse is Better? If we think Worse is Better is the way to go, then we should probably wrap the file actions API more directly, as in sham1’s proposal https://codeberg.org/sham1/pre-srfi/src/branch/main/low-level-process-manipulation.md

— Reply to this email directly, view it on GitHub https://github.com/johnwcowan/r7rs-work/issues/44#issuecomment-2470919163, or unsubscribe https://github.com/notifications/unsubscribe-auth/AANPPBXPDXIRAFLMPXI33EL2AIQXZAVCNFSM6AAAAABNXMW246VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDINZQHEYTSMJWGM . You are receiving this because you commented.Message ID: @.***>