search

mozilla / sccache

Sccache is a ccache-like tool. It is used as a compiler wrapper and avoids compilation when possible. Sccache has the capability to utilize caching in remote storage environments, including various cloud storage options, or alternatively, in local storage.
Apache License 2.0
5.85k stars 552 forks source link

sccache --stop-server does not block until the server has stopped #2237

Open Snektron opened 3 months ago

Snektron commented 3 months ago

I'm using sccache in a CI environment, where at the start of the CI job the sccache server is first started and then stopped at the end. I'm running into some issues where the CI runtime (gitlab in this case) kills the CI job before the final object file is uploaded to the cache, as sccache --stop-server runs asynchronously. For now, I've solved this by manually waiting on the sccache process by running timeout 11 tail --pid=$(pgrep sccache) -f /dev/null right after invoking --stop-server, but it would be nice if there was a built-in method to do this. For example, something like sccache --stop-server-and-wait.

just-an-engineer commented 3 months ago

Hey, I just made a PR so you can specify server_shutdown_timeout_ms from the config file. This will let you set it to a crazy high number if you really wanted to, and allow any compile job to finish. Testing on my own machine, I found that if there was no job running, it would shut down immediately upon receiving the signal. But upon a long-running job, it would either wait for it to end and then stop, or timeout (which was previously a hardcoded default of 10 seconds). Can you try it out and let me know if that fixes the issue, or if not, what's going on so I can help out more?

just-an-engineer commented 3 months ago

I marked it as a draft because I still need to add tests, which I'll do tonight. So if you want to do that today, you may need to switch over to my branch momentarily.

Snektron commented 3 months ago

Can you try it out and let me know if that fixes the issue, or if not, what's going on so I can help out more?

Well not exactly: In my case the problem was not that the upload task was exceeding the timeout (though its very useful to be able to configure that), but that there is no built-in way to block the current shell until the background server has shut down.