search

nasa / bplib

Apache License 2.0
30 stars 13 forks source link

Implement deduplication on storage #99

Closed jphickey closed 2 years ago

jphickey commented 2 years ago

When bringing bundles into storage, it should check if the bundle is already known to this storage entity. This can happen, for instance, if the bundle is using custody tracking, but the acknowledgement did not make it back to the previous custodian. In this case, the previous custodian will re-send the bundle, but since it is already present in storage it should not be stored a second time.

jphickey commented 2 years ago

Will need to tackle this next, as when testing with "bpcat" (part of PR #103) it is actually quite easy to get into situations where the same bundle is sent multiple times. These need to be consolidated at the intermediate node(s).