Hosting a Static Website on Storj: Potential Issue?

rdegges commented 2 months ago

Hi there! I'm attempting to use Storj to host a static website (my personal blog). I've been previously running it on AWS out of an S3 bucket with Cloudfront as a CDN layer, but would love to fully migrate to Storj and eventually throw an CDN in front of Storj.

I asked on the developer forum about my issue which explains the problem in more detail: https://forum.storj.io/t/problems-hosting-static-sites-with-storj/27214/6

The TL;DR version is that I've:

Created a Storj bucket for my website
Uploaded my static site files into my bucket
Used uplink share --public --readonly --dns <mycustomdomain> --url sj://<mybucket> --not-after none to make my bucket accessible as a website
I've taken the appropriate DNS records and added them to my custom domain per the uplink CLI output above
PROBLEM: After following all directions I've run into an issue where the HTML pages in my static site are rendered properly, but for some unknown reason, none of my assets (CSS, js, images, etc.) are accessible. They all return a 400 malformed request error.

Expected Behavior

Given the steps I've followed above, I'd assume that my static site should render perfectly fine and all assets should be accessible without returning a 400. This would make my site render properly.

Current Behavior

Explained this above.

Possible Solution

Not sure what the issue could be...

Steps to Reproduce

Live example (for reference: www.rdegges.com) <-- This is my static site I'm experiencing the issue with.

Context (Environment)

My main goal is just to host a simple static website on Storj. Huge fan of the service and I run a node as well, so would be nice to migrate off AWS.

Detailed Description

N/A

Possible Implementation

N/A

jtolio commented 2 months ago

Hi @rdegges !

This is interesting and confusing. Here are two example (personal) sites of mine that are set up in the exact way you're suggesting (formerly CloudFlare + S3, Hugo output, now stored in a Storj bucket).

The first one, www.jtolio.com, differs from your site in that my site is hosted out of a subfolder (oliosite is the bucket, jtolio/ is the key prefix). But I don't think that's the problem, because my other site, pub.olio.lol, is just the raw bucket with no prefix. Here's an example file, working out of the bucket: https://pub.olio.lol/pdf/challenger.pdf

Here are the TXT record setups for both:

$ dig @1.1.1.1 txt-www.jtolio.com TXT
;; ANSWER SECTION:
txt-www.jtolio.com.     300     IN      TXT     "storj-root:oliosite/jtolio" "storj-access:jxb3xcosji3cqehkiw7kyfipw5eq" "storj-tls:true"

$ dig @1.1.1.1 txt-pub.olio.lol TXT
;; ANSWER SECTION:
txt-pub.olio.lol.       300     IN      TXT     "storj-root:pub.olio.lol" "storj-access:jv4rvugdwvz62qfrcwvmdnqb2ida" "storj-tls:true"

And here is yours:

$ dig @1.1.1.1 txt-www.rdegges.com TXT
;; ANSWER SECTION:
txt-www.rdegges.com.    1799    IN      TXT     "storj-access:jupalpe3zsllmpgue7td4gr4pnja"
txt-www.rdegges.com.    1799    IN      TXT     "storj-root:rdegges-www"
txt-www.rdegges.com.    1799    IN      TXT     "storj-tls:true"

The only thing I see different in your setup and mine is perhaps that your DNS server is returning these in such a way that dig is showing multiple rows instead of a single row with multiple values? But that shouldn't be a problem either. We're still looking.

jtolio commented 2 months ago

Oops, yeah, this is a bug, thank you for reporting this. If you want a workaround right away, the fix is we accidentally stopped serving content that starts with /static/, so any other path would work :grimacing:

https://review.dev.storj.io/c/storj/edge/+/13940

rdegges commented 2 months ago

Wow! Crazy fast identification time =D Thank you.

I can now see why that's a bit hard to debug, heh. I'm not super familiar with the Storj development process, but since it's identified and I'm not in a rush, I presume a fix will be out within a few weeks? I'll probably just leave all my assets under /static in the interim as I'm lazy =p

rdegges commented 2 months ago

Just a quick update -- if you need me to reproduce it in the future, please let me know. I ended up migrating back to S3 temporarily until this is resolved. But happy to help!

pwilloughby commented 2 months ago

@rdegges The fix has been deployed if you are interested in giving it another try.

storj / edge