Support uploading/serialized crawled output to IPFS

[RangerMauve]

• [ ] WARC to CAR library
• [ ] WACZ to CAR library (with embedded WARC chunking_)
• [ ] Add code to export WACZ from crawler to CAR
• [ ] Upload CAR to IPFS with auto-js-ipfs
• [ ] Look at splitting CAR files at "max sizes"
• [ ] Paralellize uploading WARC data (multiple files at once?)

[RangerMauve]

Relevant links:

• Spec text: https://github.com/webrecorder/specs/pull/136
• Dealing with ZIp files quickly https://github.com/greggman/unzipit
• https://github.com/webrecorder/wabac.js/blob/main/src/wacz/ziprangereader.js
• New wacz processing library we might want to use: https://github.com/harvard-lil/js-wacz
• Multibase CID for identity CID: https://github.com/multiformats/multibase#multibase-table
• Golang library for doing UnixFS CAT https://github.com/anjor/unixfs-cat/blob/main/unixfs_cat.go

[RangerMauve]

Gonna try to implement a blockstore over the CAR.

Thinking of using this to ease the concurrent writes to the CAR writer stream: https://www.npmjs.com/package/await-lock