Open olizilla opened 1 year ago
@olizilla Thanks for the quick comment!
I double check and right, the read performance was related to ffplay
sending range requests. I checked with a few online players and the performance was better.
So, to summarize the current state of art:
Is that correct? @olizilla
read performance - it's good, the only issue is that web3.storage does not support range requests
Range requests are supported but they are handled by a slower code path today. We will fix this so that range requests are fast too! I can't say when, but Q1 is a very safe estimate.
write performance - it takes quite a while (5-30 min) until files are available at the expected speed
this should be fixed since we introduced freeway a few weeks ago! The slow time to availability you experienced is connected to the range-requests as well!
If you upload your files to using the web3.storage
api client (or manually to api.web3.storage) then the current implementation will make your uploads available from the w3s.link gateway as soon as you get the response ...assuming you dont send range headers, and you sent a complete DAG, which is the case when uploading regular files rather than custom IPLD graphs. If you do request it with range headers today, then the slower code path is used which also is less predictable as to when the gateway will be able to provide it.
But again, this limitation will be fixed. You will be able to have instant range requests soon! if that isn't what you find in your testing (without range headers), please do let us know!
Thanks for the explanation @olizilla
I checked again without using any range requests. And right, the download speed is ok. Still I see a slight difference between the download "just after upload" and download "a few minutes later". I tried now 3 times, uploading a set of video files and then downloading them.
Video N | Download time just after upload | Download time later a few minutes later |
---|---|---|
1 | 29.583134s | 21.553767834s |
2 | 2m50.00302425s | 1m5.003229834s |
3 | 30.002897625s | 15.000807834s |
In any case, the video is actually possible to playback just after the upload.
We recently made the w3s.link gateway go much faster for content uploaded to web3.storage, with the introduction of the freeway.
Your test-case helps identify an issue which we intend to fix! I tried out your test video with an online m3u8 player, and playback is fast, and from the headers I can see all the response are coming from freeway. At 1080p on 2x playback speed I'm able to playback the entire video without waiting. Milage may vary depending on network speed, but here is where i tested it. Let me know if you get better performance with it:
https://www.m3u8player.live/#type=m3u8&src=https%3A%2F%2Fbafybeibowt4rc4g2lahrl4e7esuwykfrf35ciid6i6lzbcz33czk3wzcjq.ipfs.w3s.link%2Fcracow_video%2Findex.m3u8
We suspect that
ffplay
may be sending range requests even though the video is chunked already. We can't yet serve range-requests via freeway... so if that's the case the video would be served via the slower path. We'll post updates on progress on the fix here https://github.com/web3-storage/gateway-lib/issues/12