search

omaha-consulting / omaha-server

Google Omaha ("Chromium Update") server
https://omaha-consulting.com
Other
209 stars 82 forks source link

Accelerate s3 #234

Closed kaji-bikash closed 6 years ago

kaji-bikash commented 7 years ago

High-speed delivery on updates from Omaha can be improved! Amazon S3 Transfer Acceleration enables fast, easy, and secure transfers of files over long distances between your client and an S3 bucket. Transfer Acceleration takes advantage of Amazon CloudFront’s globally distributed edge locations. As the data arrives at an edge location, data is routed to Amazon S3 over an optimized network path.

Enable turbocharged S3 transfer http://docs.aws.amazon.com/AmazonS3/latest/UG/enable-bucket-transfer-acceleration.html

coveralls commented 7 years ago

Coverage Status

Coverage remained the same at 82.701% when pulling 2cbc0779556bc9f7ca9841ff225ed875931d7385 on kajisaap:Accelerate-S3 into 9e468645314a1a9f5108e492fc0906b52638964f on Crystalnix:dev.

kaji-bikash commented 7 years ago

@kyakovenko after testing it in local setup, I have come to the conclusion that even if we force the S3_URL to point to s3-accelerate.amazonaws.com, the setup does not work as expected.

Django templating engines returns URLs with vanila s3 endpoint i.e. bucketname.s3.amazonaws.com. My django knowledge is as good as my google searches present me ;)

I even looked for a potential issue in underlying s3fuse itself and found following thread which also talks about the same but in s3fuse itself. For the sake of trying out, I modified the docker supervisord script on s3mount as follows

[program:s3fs]
command=/usr/bin/s3fs %(ENV_AWS_STORAGE_BUCKET_NAME)s /srv/omaha_s3 -f -ouse_cache=/tmp -ourl=http://s3-accelerate.amazonaws.com
autostart=true
autorestart=true

It mounts/works fine. But still no accelerated URL are emitted off the django templates eg. https://your.omahaserver.location/admin/omaha/version/XX/change/

I think the file omaha_server/omaha_server/s3utils.py needs tinkering. Can anybody help me?

Thanks!

Bivek commented 7 years ago

set AWS_S3_CUSTOM_DOMAIN = 'to your custom s3 accelerated url' in settings.py

kaji-bikash commented 7 years ago

It's great to find a good friend in the foreign land of PRs here 😀 Thanks @Bivek !!

As I was digging through google searches, I did find about AWS_S3_CUSTOM_DOMAIN too, I shall experiment with it and let all know the findings. This link lead me that information