search

robur-coop / builder

Scheduling build jobs on regular intervals, collecting artifacts
ISC License
13 stars 1 forks source link

builder-client observe-latest doesn't exit when there are errors in the build #45

Closed hannesm closed 1 month ago

hannesm commented 2 months ago

we observed some errors, such as: uploading failed and external package installation failed. these errors didn't show up in the builder-client, and it didn't exit (the server didn't close the connection).

hannesm commented 1 month ago

I checked the code, and this is likely related to the use of Lwt_condition.broadcast. Not sure how to properly deal with this, maybe use a Lwt_stream.t?

reynir commented 1 month ago

Hmm. I have thought about this before but for other reasons. When builder-server is busy receiving many output lines from a builder-worker lines to the client are dropped if the client can't keep up. To me this has been annoying because it's often long multi-line error messages that are dropped. In a similar manner the "end" message may be dropped.

For the output lines I understand that we may want to drop output lines so as to not have an indefinitely(*) growing queue of output lines in memory, but it seems crucial that we don't drop the "end" message. Maybe we can use a queue that is bounded for output lines but "unbounded" for "end" messages (of which there will be at most one in a well-behaving program). I'm not sure how best to implement that.

hannesm commented 1 month ago

fixed by #48 \o/