From @ProximaNova in #810, they suggested (summarized):
Could you provide an option for "ipwb index" that adds everything added from .warc under one pin instead of many pins. Use: for those who want to have as few CIDs in their pinset as possible. It wouldn't matter to those who don't care about having hundreds/thousands of pins. For now, I did this:
edit .cdxj with vim to just get CIDs, on per line, save to a different file (don't overwrite the CDXJ)
run cat f1.cdxj.cid | xargs -d "\n" sh -c 'for args do ipfs files cp /ipfs/$args /a/f1cid/$args && ipfs pin rm $args; done' _
run ipfs files ls --long /a then pin the CID you see for that
From @ProximaNova in #810, they suggested (summarized):
Could you provide an option for "ipwb index" that adds everything added from .warc under one pin instead of many pins. Use: for those who want to have as few CIDs in their pinset as possible. It wouldn't matter to those who don't care about having hundreds/thousands of pins. For now, I did this:
cat f1.cdxj.cid | xargs -d "\n" sh -c 'for args do ipfs files cp /ipfs/$args /a/f1cid/$args && ipfs pin rm $args; done' _ipfs files ls --long /athen pin the CID you see for that