widdix / mastodon-on-aws

Host your own Mastodon instance on AWS
https://cloudonaut.io/mastodon-on-aws/
136 stars 27 forks source link

Incorrect S3 Link when deploying to us-east-1 #10

Closed schuettc closed 11 months ago

schuettc commented 1 year ago

Looks like avatars, etc. are not using the correct URL.

This is the URL generated by the server: https://s3-us-east-1.amazonaws.com/mastodon-on-aws-bucket-870xpa8cwey1-bucket-af70sek4pfly/cache/accounts/avatars/109/383/759/229/326/700/original/0f056e8c0194defd.png

However, this is the actual url: https://mastodon-on-aws-bucket-870xpa8cwey1-bucket-af70sek4pfly.s3.amazonaws.com/cache/accounts/avatars/109/383/759/229/326/700/original/0f056e8c0194defd.png

schuettc commented 1 year ago

Fixed:

Updated task definition. However, I suspect that this will not work correctly in all regions but haven't tested. I am using us-east-1 and thought there was a different in S3 URLs there.

      "environment": [
        {
          "name": "S3_BUCKET",
          "value": "mastodon-on-aws-bucket-870xpa8cwey1-bucket-af70sek4pfly"
        },
        {
          "name": "S3_ENABLED",
          "value": "true"
        },
        {
          "name": "S3_ENDPOINT",
          "value": "https://mastodon-on-aws-bucket-870xpa8cwey1-bucket-af70sek4pfly.s3.amazonaws.com"
        },
        {
          "name": "S3_HOSTNAME",
          "value": "s3.amazonaws.com"
        },
        {
          "name": "S3_OVERRIDE_PATH_STYLE",
          "value": "false"
        },
        {
          "name": "S3_PROTOCOL",
          "value": "https"
        },
        {
          "name": "S3_REGION",
          "value": "us-east-1"
        },
      ],
schuettc commented 1 year ago

Seems like something still broken with S3 when uploading Avatar/Header, etc. Will keep poking at it.

schuettc commented 1 year ago

Local Images working now

"environment": [
        {
          "name": "S3_BUCKET",
          "value": "mastodon-on-aws-bucket-870xpa8cwey1-bucket-af70sek4pfly"
        },
        {
          "name": "S3_ENABLED",
          "value": "true"
        },
        {
          "name": "S3_ENDPOINT",
          "value": "https://s3.amazonaws.com"
        },
        {
          "name": "S3_HOSTNAME",
          "value": "s3.amazonaws.com"
        },
        {
          "name": "S3_OVERRIDE_PATH_STYLE",
          "value": "false"
        },
        {
          "name": "S3_PROTOCOL",
          "value": "https"
        },
        {
          "name": "S3_REGION",
          "value": "us-east-1"
        },
      ],
andreaswittig commented 1 year ago

I'm trying to understand, why this does not work for you with the default configuration. It works for me. Which AWS region are you using, @schuettc?

jbold commented 1 year ago

I am using us-east-1

@andreaswittig ⏫

compuguy commented 1 year ago

@andreaswittig Apparently the https://s3-$(REGION).amazonaws.com doesn't work in us-east-1. Hit this same exact problem early on @schuettc. Looks like you found a way around it. Now it just needs to be somehow included into the CloudFormation template. I made some different tweaks to the container paramerters to fix it in my fork under the istoleyourpw-deploy branch: https://github.com/compuguy/mastodon-on-aws

annatech commented 1 year ago

I can confirm the same issue as @schuettc.

To work around it I tried implementing this traffic flow... to no avail.

Route 53 > CloudFront > S3 (web app passthrough) > ALB

I also tried:

Route 53 > CloudFront > ALB

... and a few other iterations in configuration.

My goal there was to leverage a Lamba Edge function on the CloudFront distribution to fix the broken S3 bucket URL.

Anyway, that did not work.

Will wait for a permanent fix to the CloudFormation template.

Overall solid work, @andreaswittig !

compuguy commented 1 year ago

I made the following tweaks int he AppEnvironment key and values in order to get it to use the correct domain in us-east-1: https://github.com/compuguy/mastodon-on-aws/blob/istoleyourpw-deploy/mastodon.yaml#L286 Its essentially this change:

        AppEnvironment4Key: 'S3_HOSTNAME'
        AppEnvironment4Value: !Sub 's3.${AWS::Region}.amazonaws.com'

What I would honestly like is to just point the CloudFormation template to a file with environmental variables, or a mix of environmental variables and AWS secrets. Based on this, issue (https://github.com/cfn-modules/fargate-service/issues/27) I don't think that is happening anytime soon....

JohnCido commented 1 year ago

The exact same issue happened on my us-east-1 stack. This is the correct URL: https://[bucket-name].s3.amazonaws.com/site_uploads/files/000/000/002/@1x/[image].png And this is the URL used on the page: https://s3-us-east-1.amazonaws.com/[bucket-name]/site_uploads/files/000/000/002/@1x/[image].png

andreaswittig commented 1 year ago

I've added CloudFront to distribute the assets stored on S3. Could someone please check, wether the latest version (v0.11.0) is running fine in us-east-1?

compuguy commented 11 months ago

I think this has been fixed with the move to CloudFront @schuettc @andreaswittig. I've not had any issues with this S3 link issue since v0.11.0. This can be closed.

andreaswittig commented 11 months ago

@compuguy Thanks!