Slow Image Loading

[holdit]

Even on my fast home connection, image loading can be slow sometimes. It's not even the download speed, but it takes 2 or 3 seconds to start loading. This gets worse on 3G or slow 4G, which is a problem since a lot of people will be using this on the go, where signal might not be the best. And this is for small images. Larger sizes - which might be useful when we want to read text or something like that - takes even longer.

I had a quick look at the URLs and it seems to be hosted at S3's ap-southeast-2 (Sydney). A quick ping from my laptop:

ping -c 10 s3.ap-southeast-2.amazonaws.com
PING s3.ap-southeast-2.amazonaws.com (52.95.131.132): 56 data bytes
64 bytes from 52.95.131.132: icmp_seq=0 ttl=234 time=350.517 ms
64 bytes from 52.95.131.132: icmp_seq=1 ttl=234 time=369.815 ms
64 bytes from 52.95.131.132: icmp_seq=2 ttl=234 time=339.210 ms
Request timeout for icmp_seq 3
64 bytes from 52.95.131.132: icmp_seq=4 ttl=234 time=326.740 ms
64 bytes from 52.95.131.132: icmp_seq=5 ttl=234 time=348.050 ms
64 bytes from 52.95.131.132: icmp_seq=6 ttl=234 time=468.473 ms
64 bytes from 52.95.131.132: icmp_seq=7 ttl=234 time=332.683 ms
64 bytes from 52.95.131.132: icmp_seq=8 ttl=234 time=314.071 ms
64 bytes from 52.95.131.132: icmp_seq=9 ttl=234 time=328.323 ms

--- s3.ap-southeast-2.amazonaws.com ping statistics ---
10 packets transmitted, 9 packets received, 10.0% packet loss
round-trip min/avg/max/stddev = 314.071/353.098/468.473/43.543 ms

That's over 300ms away from me (Europe), so it explains the slow loading I'm seeing. It also explains the difficulties loading them on slow mobile networks (TLS handshake, slow connection, high ping to the server... it's going to be slow).

I guess the easy fix would be to put them behind AWS' own CDN (CloudFront)? This costs money though. And depending on the amount of traffic, CloudFront can be unnecessary expensive.

Since the project doesn't have revenues, perhaps you should look at cheaper alternatives. For example, Cloudflare R2 ( https://www.cloudflare.com/en-gb/products/r2/ ) is certainly cheaper than S3 and egress traffic is free. You don't even have to move your files right away from S3 as they have a feature that imports content as users load it from your S3 bucket. I'm just not sure if R2 requests are behind their CDN by default, but even the $20 plan gets you very far.

Bunny.net also has interesting prices, both for storage and as a CDN.

Since both are S3 compatible, using them should be easy.

I don't know if you should move away from AWS (for media storage), but I thought I'd mention these cheaper alternatives as they would allow you to either reduce costs or to speed up image loading around the world without spending more.

Anyway, while OCM works, this is something that could be improved.

[webprofusion-chrisc]

Thanks, yes the main factor in moving to Cloudflare is just the initial lift-and-shift of the data which will give us some egress fees for AWS and presumably ingress fees for Cloudflare. I haven't really looked at the cloudflare costs for this and there's always a risk when making the jump that you'll find some compatibility issue (e.g. default TLS cipher suites etc).

That said, yes we probably will move to Cloudflare eventually (I am a heavy cloudflare user for other things and the OCM API is already wrapped by a cloudflare worker). Step one would be to DNS proxy our own images.openchargemap.io as a custom domain and update all our image URLs to use that, at least then we'd get geographic caching.

[webprofusion-chrisc]

As a follow-up to this, I've just completed migrating our images (currently about 50GB over 93K files) to Cloudflare R2.

The migration itself using Cloudflare's own tool took about 10 mins, plus there were some backend changes which took a few hours to finish up.

We are currently pushing to both S3 an R2 as a backup for each other, but the API will return results pointing to https://media.openchargemap.io which is our R2 domain.

I have no idea if image downloads will be faster for you as a result, but hopefully they will be. For me they are about 1s for a 4MB file and 0.5s on repeated access. S3 is consistently around 1.5s for the same file.

[holdit]

It's faster for me too, especially on slow mobile data. The ping is also much lower: 8ms to the nearest Cloudflare POP. Thanks!

If you want to save some on the backup side of things, have a look at Backblaze B2 as a replacement for S3. Their performance is worse than S3, but they're good for backups or if used with a CDN. They support the S3 API and their prices ($0.005 per GB stored and $0.01 per GB egress) are hard to beat. They're also part of the "bandwidth alliance", so B2 <-> Cloudflare bandwidth is free. There's also Wasabi, which is part of the same "alliance" and similar to Backblaze, but I don't have any experience with them.