File descriptor leaks

[maiste]

This PR intends to fix the Unix.EMFILE that happens when too many file descriptors are opened simultaneously.

It tackles the problem at various places in the code, as suggested by @TheLortex (Thanks :pray: ):

• service/service.ml: the dispose function only terminates the process and don't close the descriptor that might stay. This is the point that was creating the issue from my tests, but it's a hypothesis. In the code version, we ensure it closes everything.
• worker/process.ml: the exec only terminate the proc in case of cancellation. By encapsulating it in the Lwt_process.with_process, we ensure it executes a close at the end at least.
• service/process.ml: the exec and pread are wrapped into a Lwt_process.with_process to ensure it also executes a close at the end.

:warning: This is a PR that has been tested with a local setup so it needs to be deployed at some point to make sure it fixes the problem.

[moyodiallo]

Thanks, this is good :+1: .

Some to complete: Those lines are where the internal workers are created. Each time a internal worker is spawn we have 2 pipes for child process and 2 other pipes for the parent process.

https://github.com/maiste/solver-service/blob/f125f8bc97e20e027920fc1052ee69894c9b38bc/service/main.ml#L106 https://github.com/maiste/solver-service/blob/f125f8bc97e20e027920fc1052ee69894c9b38bc/service/main.ml#L73

[maiste]

Thanks, this is good +1 .

Some to complete: Those lines are where the internal workers are created. Each time a internal worker is spawn we have 2 pipes for child process and 2 other pipes for the parent process.

https://github.com/maiste/solver-service/blob/f125f8bc97e20e027920fc1052ee69894c9b38bc/service/main.ml#L106 https://github.com/maiste/solver-service/blob/f125f8bc97e20e027920fc1052ee69894c9b38bc/service/main.ml#L73

The internal worker should close its connections when it calls dispose here.

[moyodiallo]

Thanks @maiste and @TheLortex. The PR is tested with more pressure using https://github.com/ocurrent/solver-service/pull/41 and no crash noticed.

~We can revert this PR now https://github.com/ocurrent/ocaml-ci/pull/742 @tmcgilchrist.~