Adding stream compression to retrievals

[RobQuistNL]

Checklist

• [X] This is not a new feature or an enhancement to the Filecoin protocol. If it is, please open an FIP issue.
• [X] This is not brainstorming ideas. If you have an idea you'd like to discuss, please open a new discussion on the lotus forum and select the category as Ideas.
• [X] I have a specific, actionable, and well motivated feature request to propose.

Lotus component

• [ ] lotus daemon - chain sync
• [ ] lotus miner - mining and block production
• [ ] lotus miner/worker - sealing
• [ ] lotus miner - proving(WindowPoSt)
• [ ] lotus miner/market - storage deal
• [X] lotus miner/market - retrieval deal
• [X] lotus miner/market - data transfer
• [X] lotus client
• [ ] lotus JSON-RPC API
• [ ] lotus message management (mpool)
• [ ] Other

What is the motivation behind this feature request? Is your feature request related to a problem? Please describe.

Knowing that there's quite some data that can very well be compressed (e.g. the Genome AD dataset) using tools like ZStandard (we've hit 90% compression on some of those .car files), bandwidth would greatly benefit from being able to compress the datastream coming out of retrieval deals.

Describe the solution you'd like

ZST could implemented as it supports streaming - if its used on both sides, a much higher throughput can be reached.

This will especially be useful as we're looking into the Evergreen project right now.

Describe alternatives you've considered

No response

Additional context

No response

[ribasushi]

@dirkmc @nonsense this might be of interest...?

[RobQuistNL]

@LexLuthr (just a nice enhancement which would save a lot of people a lot of time and money)