Closed ViralBShah closed 4 years ago
Some quick stats for the last month of June:
Domain | GB/day |
---|---|
julialangnightlies-s3 | 156.2 |
julialang-s3 | 129.7 |
julialangmirror-s3 | 39.1 |
Everything else is peanuts compared to these three. I'm really surprised by the nightlies numbers; I'm guessing this is predominantly CI, as I just don't see us pushing 150 GB of nightlies per day to typical users.
If our CI (Travis, Appveyor, etc...) can be served by a cheaper source, (e.g. interposing an AWS-local source for AWS-hosted CI services to make use of the free within-availability zone bandwidth) we can likely cut this down by a lot.
Appveyor does not run on AWS (they are on rackspace). I am not sure where Travis and Circle run. I suspect we need to push local binaries to all the places. Maybe cut down circle, if we no longer need it with Travis being stable (although that can change any time)?
Oh hey, old issue! We have successfully implemented our own cloud-native caching system for Julia downloads, and we're well on our way to doing the same for packages. Issue, consider thyself closed!
We surpassed our comped download on Fastly this month (> 9TB).
There are probably opportunities to save bandwidth by doing things like optimizing on CI downloads (by caching binaries directly into the CI systems). We should see if there are other things we can do.