rust-lang / jobserver-rs

Apache License 2.0
69 stars 40 forks source link

New jobserver type in GNU Make 4.4 #47

Open bjorn3 opened 1 year ago

bjorn3 commented 1 year ago

Docs: https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver Release announcement: https://lwn.net/Articles/913253/

On systems that support it GNU Make 4.4 will now use --jobserver-auth=fifo:PATH where PATH points to a named pipe rather than --jobserver-auth=R,W. I haven't checked if this breaks anything, but based on looking at the code it seems like it will cause the jobserver to be ignored entirely.

kaspar030 commented 1 year ago

I can confirm that the new (default since GNU make 4.4) fifo method makes this crate ignore the jobserver. That unfortunately includes Cargo.

Here's a reproducer: https://github.com/kaspar030/jobserver-rs-fifo-test

as a workaround, use --jobserver-style=pipe as top-level make option.

Here's the output from above's test application:

``` ❯ make -j2 echo $MAKEFLAGS -j2 --jobserver-auth=fifo:/tmp/GMfifo1775897 cargo build Finished dev [unoptimized + debuginfo] target(s) in 0.01s target/debug/jobserver-rs-fifo-test MAKEFLAGS= -j2 --jobserver-auth=fifo:/tmp/GMfifo1775897 MFLAGS=-j2 --jobserver-auth=fifo:/tmp/GMfifo1775897 None jobserver-rs-fifo-test on  main ❯ make -j2 --jobserver-style=pipe echo $MAKEFLAGS -j2 --jobserver-auth=3,4 --jobserver-auth=-2,-2 cargo build Finished dev [unoptimized + debuginfo] target(s) in 0.01s target/debug/jobserver-rs-fifo-test MAKEFLAGS= -j2 --jobserver-auth=3,4 --jobserver-auth=-2,-2 MFLAGS=-j2 --jobserver-auth=3,4 --jobserver-auth=-2,-2 Some(Client { inner: Client { read: File { fd: 3, path: "pipe:[31836913]", read: true, write: false }, write: File { fd: 4, path: "pipe:[31836913]", read: false, write: true } } }) ```
weihanglo commented 1 year ago

49 probably resolves the compatibility of new GNU Make 4.4. For creating a new client in fifo style, the idea is still brewing. If anyone has a great design of that, I am willing to listen and help :)

the8472 commented 12 months ago

For safety/reliability reasons this should become the preferred jobserver protocol in the future since it avoids mucking with file descriptors based on environment variables where inherited environment variables and inherited descriptors can get out of sync.

anacrolix commented 7 months ago

Could you be more specific about the workaround? How do I pass --jobserver-style=pipe? I have tried MAKEFLAGS=--jobserver-style=pipe cargo build but it didn't fix the problem for me.

weihanglo commented 7 months ago

as a workaround, use --jobserver-style=pipe as top-level make option.

See --jobserver-type in GNU make manual.