search

tagbase / tagbase-server

tagbase-server is a data management web service for working with eTUFF and nc-eTAG files.
https://oiip.jpl.nasa.gov/doc/OIIP_Deliverable7.4_TagbasePostgreSQLeTUFF_UserGuide.pdf
Apache License 2.0
7 stars 2 forks source link

Implement signature calculation for each file #239

Closed lewismc closed 1 year ago

lewismc commented 1 year ago

We can prevent duplicate file ingestions by calculating a signature for each file. The signature would be based off of the entire file bytes. We can store this signature in the submission table. When a ingestion request is being processed the first thing we should do it calculate the signature and determine whether those bytes have been processed before.

lewismc commented 1 year ago

Implemented in https://github.com/tagbase/tagbase-server/pull/264