We noticed when launching a large number of applications on a single machine, intermittently OutOfMemoryExceptions would be raised, caused by operating system user-threads being exhausted (especially on Linux).
Looking at before and after thread dumps of the application (ie: test) that launches the large number of applications, it seems that the InputRedirector threads that redirect stdin from the platform to the said applications never terminate, even though the underlying applications themselves are being correctly cleaned up. Although an attempt is made to interrupt these threads when closing an application, the threads may be blocked (and not respond to being interrupted), thus leaking them.
With a large enough number of applications being launched, especially in a long running rolling restart manner, this may cause exhaustion of operating system user-space threads and thus cause OutOfMemoryExceptions when launching applications.
We noticed when launching a large number of applications on a single machine, intermittently OutOfMemoryExceptions would be raised, caused by operating system user-threads being exhausted (especially on Linux).
Looking at before and after thread dumps of the application (ie: test) that launches the large number of applications, it seems that the InputRedirector threads that redirect stdin from the platform to the said applications never terminate, even though the underlying applications themselves are being correctly cleaned up. Although an attempt is made to interrupt these threads when closing an application, the threads may be blocked (and not respond to being interrupted), thus leaking them.
With a large enough number of applications being launched, especially in a long running rolling restart manner, this may cause exhaustion of operating system user-space threads and thus cause OutOfMemoryExceptions when launching applications.