ARSN-370: fix memory leak in MongoDBReadStreams

[williamlardier]

The MongoDBReadStreams are not properly destroyed in both the Bucket V1 and V0 cases. In the V1 case, only the pipe-ed stream, the Transform one, is cleaned. In the V0 case, we directly call the callback without properly cleaning the stream, leaving open, in both cases, the mongodb cursors, that in turn affect the mongos memory consumption.

The fix was tested end2end:

Before (with k8s memory limits, otherwise it uses up to 100% of the cluster RAM): Causing heap overflows in cloudserver pods and high MongoS memory usage due to open cursors.

After:

[bert-e]

Hello williamlardier,

My role is to assist you with the merge of this pull request. Please type @bert-e help to get information on this process, or consult the user documentation.

Status report is not available.

[bert-e]

Waiting for approval

The following approvals are needed before I can proceed with the merge:

• the author

• 2 peers

[codecov-commenter]

Codecov Report

Merging #2177 (17b5bbc) into development/8.1 (af34571) will increase coverage by 0.05%. The diff coverage is 100.00%.

@@                 Coverage Diff                 @@
##           development/8.1    #2177      +/-   ##
===================================================
+ Coverage            64.28%   64.34%   +0.05%     
===================================================
  Files                  210      210              
  Lines                16381    16391      +10     
  Branches              3320     3320              
===================================================
+ Hits                 10531    10547      +16     
+ Misses                5835     5829       -6     
  Partials                15       15              

Files	Coverage Δ
...orage/metadata/mongoclient/MongoClientInterface.js	63.18% <100.00%> (+0.73%)	:arrow_up:

... and 1 file with indirect coverage changes

[bert-e]

Incorrect fix version

The Fix Version/s in issue ARSN-370 contains:

• 8.1.112

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

• 8.1.113

Please check the Fix Version/s of ARSN-370, or the target branch of this pull request.

[williamlardier]

/approve

[bert-e]

In the queue

The changeset has received all authorizations and has been added to the relevant queue(s). The queue(s) will be merged in the target development branch(es) as soon as builds have passed.

The changeset will be merged in:

• :heavy_check_mark: development/8.1

The following branches will NOT be impacted:

• development/6.4
• development/7.10
• development/7.4
• development/7.70

There is no action required on your side. You will be notified here once the changeset has been merged. In the unlikely event that the changeset fails permanently on the queue, a member of the admin team will contact you to help resolve the matter.

IMPORTANT

Please do not attempt to modify this pull request.

• Any commit you add on the source branch will trigger a new cycle after the current queue is merged.
• Any commit you add on one of the integration branches will be lost.

If you need this pull request to be removed from the queue, please contact a member of the admin team now.

The following options are set: approve

[bert-e]

I have successfully merged the changeset of this pull request into targetted development branches:

• :heavy_check_mark: development/8.1

The following branches have NOT changed:

• development/6.4
• development/7.10
• development/7.4
• development/7.70

Please check the status of the associated issue ARSN-370.

Goodbye williamlardier.