nasa / bplib

Apache License 2.0
27 stars 13 forks source link

Improve discretion in block allocation for low memory situations #137

Closed jphickey closed 2 years ago

jphickey commented 2 years ago

When memory is running low, not all blocks should be considered equal from an allocation perspective. In order to reduce memory pressure, bundles must be forwarded to the next custodian, but in order to do this blocks need to be temporarily loaded from a file into RAM. Additionally, the DACS bundle needs to be received in order to delete the bundle from the local index, which takes additional buffers to do so.

If all memory is used for stored bundles, then the system will basically deadlock - no bundles can be forwarded nor can any DACS be received to reduce the memory use.

To avoid this, the pool reserves a cushion, but the logic is not ideal in that it only affects primary block allocation, and it treats data coming in from an application the same as bundles received from a CLA (former will only increase pool usage, but the latter of which could be DACS which will reduce pool usage).

There should be a more fine-grained priority level when allocating blocks from the pool, such that operations that may (eventually) reduce memory pressure can be given priority over those which will increase memory pressure.

jphickey commented 2 years ago

Closed by mistake, not merged yet...

jphickey commented 2 years ago

Now has been merged