search

sciencehistory / terraform_scihist_digicoll

0 stars 0 forks source link

Video-derivatives cloudfront uses new header policy that forces CORS #73

Closed jrochkind closed 7 months ago

jrochkind commented 7 months ago

Ref: https://github.com/sciencehistory/scihist_digicoll/issues/2523

Our video derivatives files need to be served with CORS headers appropriate for use with video.js. We're having a lot of trouble getting this to work reliably.

Previously we were trying to: a) set up proper CORS headers on the S3 bucket, and b) make sure Cloudfront was appropriately proxying these headers.

This approach matched the documentation we found here for Configuring CloudFront to respect CORS settings

However, we are having trouble making that work reliably. Sometimes CloudFront ends up caching and delivering a response without CORS files. We don't know what is going wrong with which of either a or b above. This happened once in Oct 2023 at https://github.com/sciencehistory/scihist_digicoll/issues/2404 . We thought we fixed this -- but then despite the changed config, it happened again in Jan 2024 at https://github.com/sciencehistory/scihist_digicoll/issues/2523

Looking around, we saw that there was another possible CloudFront solution introduced in 2021 and not doc'd in the above guide: We can have CloudFront force CORS headers, regardless of what the underlying source HTTP server is doing. Per here and then here

There are some AWS"managed policies" for forcing CORS headers, but since we already have a custom header policy to force far-future cache headers -- and you can only use one response header policy at a time -- we coudln't juse use the managed policy, but had to create our own custom policy that does what the managed policy did, along with the additional behavior we want.

That is what we do here.

At the moment i have not yet invalidated the cache on production for the problem video. I am interested to see, if we merge and apply this to prod -- will that immediately fix the problem I can currently reproduce with https://digital.sciencehistory.org/works/9nhk6as -- without need for cache invalidation?

If so, that will be a good sign that we've finally fixed it.

jrochkind commented 7 months ago

This branch is already applied in terraform staging workspace, where it seems to work.

jrochkind commented 7 months ago

I'll remark for the record that it's intentional we are leaving the config in for S3 to produce CORS headers directly too -- even though it shoudn't be necessary with this configuration, it leaves it open to switching back to direct S3 or other configurations which might need CORS headers from S3 directly.

jrochkind commented 7 months ago

Checked right before applying in prod to verify the problem was still there.

Applied in production.

It is fixed. Even with no cache invalidation.

So I am optimistic this change will have solved the problem for good! 🧿