Added support to configure routing rules (redirects) for s3 bucket from config file.

[JangoSteve]

I have yet had time to create any tests (and I probably won't any time soon), but I wanted to go ahead and create the pull request as a reminder, and in case it helps anyone else.

I really like using the jekyll-s3 gem to manage our jekyll-generated static site deployment on S3. I think it's awesome being able to contain the deployment configuration in the repo.

In developing one of our sites though, I had to change the bucket a couple times, since the bucket name must match the desired CNAME for the custom domain, and we changed the subdomain a couple times. Doing so made me realize that there's an important part to the S3 website configuration that I had to re-do manually through the AWS console for each new bucket, and that is the routing (aka redirect) rules.

Once I realized that the configure-s3-website gem uses the PUT Bucket Website API, and that the routing rules could be specified by the same API endpoint, it didn't seem like much of a stretch to add routing rules support to the gem.

With this pull request, for existing jekyll-s3 projects with no routing rules, one line is added to the output of the configure-s3-website command:

$ configure-s3-website --config-file _jekyll_s3.yml
Bucket www.example.com now functions as a website
Bucket www.example.com is now readable to the whole world
No redirects to configure for www.example.com bucket.

But now we can optionally include routing rules in our _jekyll_s3.yml config file, like this:

s3_id: S3_SECRET_ID
s3_secret: S3_SECRET_KEY
s3_bucket: S3_BUCKET
routing_rules:
  - condition:
      key_prefix_equals: blog/some_path
    redirect:
      host_name: blog.example.com
      replace_key_prefix_with: some_new_path/
      http_redirect_code: 301
  - condition:
      key_prefix_equals: blog/another_old_path/
    redirect:
      host_name: blog.example.com
      replace_key_prefix_with: another_new_path/
      http_redirect_code: 301
  - condition:
      key_prefix_equals: blog/my_old_project/
    redirect:
      host_name: projects.example.com
      replace_key_prefix_with: my_old_project/
      http_redirect_code: 301

Now the redirects will automatically be configured for us on the S3 bucket:

$ configure-s3-website --config-file _jekyll_s3.yml
Bucket www.example.com now functions as a website
Bucket www.example.com is now readable to the whole world
3 redirects configured for www.example.com bucket.

Any of the other condition and redirect keys can be used from the API redirect documentation linked above, e.g. replace_key_with instead of replace_key_prefix_with, etc., it will just translate the underscore-formatted keys in the yaml file to the camelcase-formatted keys in the XML string that gets sent with the API call.

So the above yaml config sends this with the API call:

          <WebsiteConfiguration xmlns='http://s3.amazonaws.com/doc/2006-03-01/'>
            <IndexDocument>
              <Suffix>index.html</Suffix>
            </IndexDocument>
            <ErrorDocument>
              <Key>error.html</Key>
            </ErrorDocument>
            <RoutingRules>

              <RoutingRule>

              <Condition>
                <KeyPrefixEquals>blog/some_path</KeyPrefixEquals></Condition>
              <Redirect>
                <HostName>blog.example.com</HostName>
                <ReplaceKeyPrefixWith>some_new_path/</ReplaceKeyPrefixWith>
                <HttpRedirectCode>301</HttpRedirectCode></Redirect>
              </RoutingRule>

              <RoutingRule>

              <Condition>
                <KeyPrefixEquals>blog/another_old_path/</KeyPrefixEquals></Condition>
              <Redirect>
                <HostName>blog.example.com</HostName>
                <ReplaceKeyPrefixWith>another_new_path/</ReplaceKeyPrefixWith>
                <HttpRedirectCode>301</HttpRedirectCode></Redirect>
              </RoutingRule>

              <RoutingRule>

              <Condition>
                <KeyPrefixEquals>blog/my_old_project/</KeyPrefixEquals></Condition>
              <Redirect>
                <HostName>projects.example.com</HostName>
                <ReplaceKeyPrefixWith>my_old_project/</ReplaceKeyPrefixWith>
                <HttpRedirectCode>301</HttpRedirectCode></Redirect>
              </RoutingRule>

            </RoutingRules>
          </WebsiteConfiguration>

[JangoSteve]

FYI, it looks like the Travis build fails with this simply due to the added lines to the output.

[laurilehmijoki]

Hello, your idea is nice.

Would you like to become a contributor for this GH repo as well as the gem configure-s3-website?

Due to my other commitments, I don't often have time to concentrate on this project.

[JangoSteve]

Oh sorry, I didn't see your response before. Sure!

[laurilehmijoki]

Thanks for the great implementation! It's included in version 1.2.0.

I added a Cucumber example, you can find it here: https://github.com/laurilehmijoki/configure-s3-website/commit/d2d25fa658d0ff4771c1edbfa7673ad78073307e.

This feature will be useful for jekyll-s3 users, too.

[mikeedla]

Is there a way to reroute all files ending in .php to files ending in .html with the same url for example i recently am changing from my php site to amazon static my php site would have forexample example.com/games/pacman.php and now that example.com will redirect to www.example.com using route 53 i want www.example.com/games/pacman.php to redirect to www.example.com/games/pacman.html

[JangoSteve]

You could create a redirect rule such as this:

  - condition:
      key_prefix_equals: games/pacman.php
    redirect:
      host_name: www.example.com
      replace_key_prefix_with: "games/pacman.html"
      http_redirect_code: 301

But it's hardcoded, so you'd need one for each page you want to redirect. I don't know if Amazon's S3 rerouting rules support any type of pattern or regex matching (I haven't seen any examples online, and I haven't tried it myself). See their redirect docs for how they say to do the routing (and note that none of their examples have pattern matching, but they also don't say it's not supported).

[mikeedla]

thank you for your quick response, unfortunately it looks like i will have to do this about 500 times for now :(, i will keep searching though.

[JangoSteve]

I haven't tested this, so it may still need a bit of debugging, but maybe write yourself a little ruby script like this?

File.open('_jekyll_s3.yml', 'w') { |file|
  Dir.glob('/path/to/dir/*.php') do |php_file|
    rule = <<EOS
  - condition:
      key_prefix_equals: path/to/dir/#{php_file}
    redirect:
      host_name: www.example.com
      replace_key_prefix_with: "#{php_file.gsub('.php', '.html')}"
      http_redirect_code: 301
EOS
    file.write(rule)
  end
}

They'd still be all hardcoded, but at least you wouldn't have to write the 500 rewrite rules yourself.

[mikeedla]

so i tried to write the 500 rewrite rules, because i'm not a great coder, unfortunately they limit you to 50 redirects, ugh. I have no idea how to do ruby, let alone doing any scripting on s3, i always do it via web or via cyber duck any suggestions?

[JangoSteve]

Oh hmmm, if they limit you to 50 rewrites, then I have no idea. If it were me, I'd open a support ticket with AWS and ask how you could accomplish those rewrites.