kaulson commented 7 months ago

Refactored ConcurrentWrapper to ensure proper closure of file descriptors and restoration of standard output/error streams using context managers, enhancing resource management and reliability.

Description

In the original implementation, file descriptors for stdout and stderr were opened but not explicitly closed. This could potentially lead to resource leaks, especially if the wrapper is used to spawn many instances. In this version, the with statement ensures that the file descriptors are properly closed once the block is exited, even if exceptions occur. Additionally the standard output (sys.stdout) and error (sys.stderr) were redirected to the files but were not restored to their original state after the runnable function finished executing. This means that any subsequent prints or error messages in the same process (or thread, depending on the execution environment) would continue to be redirected to the last files opened, which is likely unintended behavior. This MR addresses this by storing the original sys.stdout and sys.stderr, then restoring them after executing the runnable function, ensuring that the redirection is temporary and scoped only to the execution of runnable.

I found these problems when I identified an issue where the last line of output was sometimes not captured. This should be due to to the standard output and error buffers not being explicitly flushed.

Related Issue (if any)

n/a

Motivation and Context

The current version sometimes fails to capture all output of a runnable. The added tests show which problems may occur.

How Has This Been Tested?

I added a few tests to tests/test_concurrent_wrapper.py. I test if delayed output is captured, the descriptors are closed and that the stdout is restored.

If anything is missing or unclear, please let me know.

facebook-github-bot commented 7 months ago

Hi @kaulson!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

robertodessi commented 7 months ago

Hello @kaulson,

Thanks for your PR. It looks good to me but unfortunately I believe this repo is not actively maintained or monitored by anyone at Meta. Someone should fork it so that it can be developed should someone wish to do so.