Improved directory listing experience with CloudFront Functions

[gw0]

In a self-hosted scenario, it is a bit off that one has to expose the S3 website (to host index.html) and S3 REST API endpoints (to get the XML listing), then configure CORS rules accordingly. A much cleaner solution can be accomplished if everything is behind CloudFront (this is also the only way to use your own domain with valid TLS certificates)...

You only need to expose the S3 REST API (disable website mode) to your CloudFront distribution (using CloudFront Origin Access Controls), upload a correctly configured AWS S3 Bucket Browser into /s3-browser/, and use the following CloudFront Function to rewrite a few URLs. To support directory index listing I also modified to get the pathPrefix from "document.location.pathname" as a fallback.

-      bucketUrl: undefined,
-      keyExcludePatterns: [/^index\.html$/, /^Urban_3D_Challenge\//], // matches againt object key relative to rootPrefix
-      bucketMaskUrl: undefined,
+      bucketUrl: '/list-objects',
+      keyExcludePatterns: [/^s3-browser\//],
+      bucketMaskUrl: '/',

-        window.onpopstate = () => this.pathPrefix = decodeURIComponent(window.location.hash).replace(/^#/, '')
+        window.onpopstate = () => this.pathPrefix = decodeURIComponent(window.location.hash).replace(/^#/, '') || decodeURIComponent(window.location.pathname).replace(/^\//, '')

/**
 * Rewrite root and directory index to an S3 bucket browser web app using AWS CloudFront Functions.
 */

var s3ListingUri = "/";
var s3BrowserUri = "/s3-browser/index.html";

function handler(event) {
  var request = event.request;

  if (request.uri == '/list-objects') {
    // Request for XML listing points to the S3 root listing endpoint
    //   https://bucket.domain/list-objects
    request.uri = s3ListingUri;

  } else if (request.uri.endsWith('/')) {
    // Request for root or directory index points to the S3 browser
    //   https://bucket.domain/
    //   https://bucket.domain/my/path/
    request.uri = s3BrowserUri;
  }

  return request;
}

[qoomon]

Thanks for this hosting idea.

[x3LPh0r]

Hi @gw0 thank you for the insight on this. I am looking to utilize bucket browser on a private bucket and came across this post. I performed your recommendations above with the index code edits and created the cf function and applied it to the viewer request, however I get the same error(s) I've been having trying to get a private bucket to work:

Bucket URL /list-objects is not a valid bucket API URL, response does not contain <'ListBucketResult'><'Delimiter'> tag.

Curious if I'm missing something with your recommendation or if there are any other changes that need to take place to get this to work successfully. Appreciate it.

[gw0]

Bucket URL /list-objects is not a valid bucket API URL, response does not contain <'ListBucketResult'><'Delimiter'> tag.

It seems your CF Function is not enabled correctly. The purpose of the CF Function is to rewrite the URL /list-objects to /. The response you get when listing /list-objects with the CF Function enabled should be the same as the response you get when listing / with CF function disabled.

[x3LPh0r]

Thank you for the response. Interesting. I've tried I feel like every combination of configuration/settings possible and I get the same error message every time. Perhaps something has changed where this does not work today as it did previously. I also tried other methods entering the cloudfront domain in the bucket url setting as another person had previously which throws the same error just with the domain.

Few screenshots attached. I tried both legacy cache settings and the new recommended with the appropriate headers. I tried OAI first (as I was using it from the previous mentioned test from another person) and switched to OAC and updated the policies for both listing and getting the bucket. I tried both js-2.0 and js-1.0 for the cf function. I do see invocations present in the function metrics. Continuosly the same error present on the web page gui. Seems like the only way I can get it to work is making it public without OAC running it as a static website (which of course I want to avoid).

Do you have this functioning today on your end? Thanks again.

[gw0]

Do you have this functioning today on your end?

Yes, everything works.

I don't know what is wrong, but it will never work until you fix it to behave like this: The response you get when listing /list-objects with the CF Function enabled should be the same as the response you get when listing / with CF function disabled.

[x3LPh0r]

Alright, for anyone else coming here and wanting this to work properly, this is the problem I was having with this and trying other configurations. I was specifying the default root object in Cloudfront. This causes the bucket browser to break and only work when utilizing an s3 static website public endpoint. To get this to work make sure you delete/remove the default root object in your Cloudfront distribution. I was banging my head on this, driving myself crazy. Knew it had to be a simple/small fix. Everything looked correct! Works like a charm deleting the default root object from Cloudfront.

Also with testing this, note that if you have a bucket that you interact with frequently you will need to initiate invalidations in Cloudfront to update the bucket browser GUI. Invalidating just the index.html will not reflect any changes, but invalidating the entire bucket /* will. In my instance, I deleted an object and uploaded an object. Neither the new object appeared nor the old object disappeared. I will have to work on a function next that invalidates the bucket as new objects are added and objects are removed from the bucket.

[MatthewCloudza]

Hi @x3LPh0r would like to know a bit more how you got this working please, really need help. Not sure if my settings are correct Any way we could have a virtual call perhaps?

[x3LPh0r]

Hi @x3LPh0r would like to know a bit more how you got this working please, really need help. Not sure if my settings are correct Any way we could have a virtual call perhaps?

Feel free to email me and we can discuss further there.