search

tech-greedy / singularity

New node software for large-scale clients with PB-scale data onboarding to Filecoin network
Other
38 stars 18 forks source link

Root CID of DataSet #186

Closed frank-ang closed 1 year ago

frank-ang commented 1 year ago

Is your feature request related to a problem? Please describe. Singularity index should have a root CID of the overall DataSet. Permit unifying all the CAR within the DataSet under a single root IPLD.

Describe the solution you'd like Index repair with DataSet-level encapsulating root CID. Enable public IPFS browser traversal. Support incremental storage use-case, e.g. appending CAR to existing DataSet index. Ref issue https://github.com/tech-greedy/singularity/issues/185

Additional context split-then-car vs car-then-split opinion from @anjor

anjor commented 1 year ago

I believe https://github.com/tech-greedy/singularity/pull/184 addresses this.

frank-ang commented 1 year ago

Probably need (manual) integration testing to verify functionality, beyond unit tests.

xinaxu commented 1 year ago

This is done, with E2E testing.