This adds an explicit cleanup on thread pools since they will not be destroyed until the GC is called. In some cases, this is not quick enough and the uploaded exhausts the number of available threads of the user.
Fixes #23 Azure table uploader spawning too many threads
Type of change
Please delete options that are not relevant.
[X] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update
How Has This Been Tested?
I have ran this on current high workload job, and checked jstack. There was a normal number of threads (~30) instead of the several thousand before.
I will manually finish the hotfix after the PR is approved
Description
This adds an explicit cleanup on thread pools since they will not be destroyed until the GC is called. In some cases, this is not quick enough and the uploaded exhausts the number of available threads of the user.
Fixes #23 Azure table uploader spawning too many threads
Type of change
Please delete options that are not relevant.
How Has This Been Tested?
I have ran this on current high workload job, and checked
jstack. There was a normal number of threads (~30) instead of the several thousand before.I will manually finish the hotfix after the PR is approved