search

webrecorder / wacz-uploader

A straightforward single page application for uploading your WACZ archives to IPFS
GNU Affero General Public License v3.0
5 stars 0 forks source link

Use content aware chunking for the wacz files to deduplicate #17

Closed RangerMauve closed 1 year ago

RangerMauve commented 1 year ago

Before we upload archives, we should generate chunk offsets based on the zip file structure and then use that to generate a CAR file with chunks which will get streamed right to w3s.storage and the such.

RangerMauve commented 1 year ago

I think this is a duplicate of #3