nasa / bplib

Apache License 2.0
27 stars 13 forks source link

Add queue limits to flow/subqueue structures #88

Closed jphickey closed 2 years ago

jphickey commented 2 years ago

The maintenance/data pump functions should not be able to fill a queue indefinitely. For example, a CLA that has become "up" may, but certainly may not, be able to transfer all the stored data that is routed through it. So in particular, a storage service that is filling the queue should implement some work limits, where it fetches some number of bundles from storage, but then yields to allow those bundles to be pushed through the system, before resuming work to fetch more bundles.

A basic prerequisite of implementing something like this is to put depth limits on the various flow queues.