offspot / imager-service

Create Kiwix Hotspot microSD cards online
https://imager.kiwix.org/
GNU General Public License v3.0
13 stars 6 forks source link

Allocate a bucket near end-user to improve download speed/quality #266

Open Popolechien opened 3 years ago

Popolechien commented 3 years ago

I ran a quick check on the last 40 transactions (20 full cardshop access / 20 Wikipedia downloads) and got the following geographic distribution:

US: 65% EU: 15% Others (India, UAE, Australia): 20% I could not see much variance in terms of image size between regions. The above also does not cover our own image generation/downloads in Switzerland.

Having 2/3 of our users in the US, it does make sense that most of our resources are based there. But would there be a way to better allocate buckets so that people in Europe are better served in terms of download speed?

rgaudin commented 3 years ago

We could move to the US-east instead of west

rgaudin commented 3 years ago

But I would expect the bottleneck to be un transatlantic transit

Popolechien commented 3 years ago

Most US users seem to be Central/East Coast, so that can't hurt.

rgaudin commented 3 years ago

For cardshop users, we could have one bucket per region and ask user to choose the region to upload to at Order time. We could choose the default based on user IP address. Those images would be stored in a single bucket.

For pre-made images, we could upload them to different region-based buckets. That would be duplication of storage (and cost) but we don't have too many and we would only support a selection of regions (eu-central and us-central ?) The torrent would include both webseeds and thus the client would get the fastest without even choosing. As for HTTP, we could share a link that would get region based on IP and redirect to closest bucket.

This would require some work though.

FYI, list of wsabi regions:

kelson42 commented 4 months ago

For cardshop users, we could have one bucket per region and ask user to choose the region to upload to at Order time. We could choose the default based on user IP address. Those images would be stored in a single bucket.

No selectbox please.