Fix/Improve Thumbor Support and Documentation

[kylemarsh]

I haven't found any documentation for it yet (maybe I'm just missing it?) and I haven't been able to make it work yet: the URLs 50mm generates seem to be invalid (at least given the stock configuration). For instance, when I enable thumbor my image links look like this (and I haven't gotten them to work yet):

https://localhost:8888/GOOP/800x0/smart/prefix/to/photo.jpg

But the thumbor project's examples look like this:

http://localhost:8888/GOOP/800x0/smart/http://bucket.objectstore.com/prefix/to/photo.jpg

Also, 50mm is generating HTTPS links by default for thumbor and I'm not sure if there's a way to disable that or if I need to actually get a legitimate certificate for my thumbor service. As suggested in Pull Request #18, Let's Encrypt is probably a completely legitimate option -- I haven't entirely grokked yet how 50mm + thumbor interact, so I'm just trying everything with localhost until I get it sorted out...and to that end, it might be nice to have a development mode that generates insecure links, even if we don't allow that in production.

[arahayrabedian]

So I'm a little rusty here - but I'll answer the part I can right now WRT the URLs: The premise behind 50mm is that you have an S3 bucket (usually private, but I don't see why this should be enforced) with photos in it. 50mm makes S3 API requests to get a list of the available photos then uses the same key to generate a URL (possibly of a resizing service) that uses that same S3 bucket as it's backing store.

In the case of imgix this is what I remember to be happening, this is also how I personally use 50mm with thumbor+cloudfront, standalone thumbor (with a plugin?) also supports S3 as a backing store.

The method you want to use is not one that I had considered when implementing the thumbor support since the implication is that you want access a resource you have privileged access to (because you have S3 API access to it since you're using 50mm) from the 'outside' (asking your thumbor server to fetch via http).

That said, I don't see any reason (again, I'm rusty here) why this method can't be supported, however, you'll still need to 'own' the bucket (since 50mm needs to make S3 API requests to get photo listings)

As for forcing https for thumbor, I don't recall there being anything that forces this to happen, I'm going to give it a shot locally to see if I can reproduce it. I'll let you know in a bit.

[arahayrabedian]

@kylemarsh the assumption with using thumbor as a resizing service was to be using this plugin: https://github.com/thumbor-community/aws, their readme provides a little info about why that would be one route to use with 50mm (TL;DR: that you don't want your originals exposed raw to the public - this is a preference, not a law)

[kylemarsh]

Oh! I didn't realize that you could point thumbor at a bucket like that. Cool. Do you have an example thumbor config that you could sanitize and share as an example? What you described is what I was thinking/hoping would happen, but I didn't get far enough into the configuration, I guess.

[arahayrabedian]

I was playing around with it myself to figure out your https prefix issue, you're going to want to set CanonicalSecure to 0 to avoid https links being generated I think - I was reminded of this locally myself just now.

As for a sample config working with standalone thumbor (which talks to S3 in the background, not over http(s)):

[DEFAULT]
Domain = localhost:8080
CanonicalSecure = 0
BucketRegion = eu-west-1
BucketName = your-s3-bucket-name
ResizingService = thumbor
ResizingServiceSecret = secret  # shared secret between 50mm and thumbor (for URL signing to prevent your thumbor being abused as an image proxy), make sure thumbor knows about this.
BaseUrl = http://localhost:8081  # where your local thumbor listens
AWSKeyId = <your_id_here>
AWSKey = <your_secret_here
SiteTitle = 50mm Gallery
MetaTitle = 50mm Gallery
HasAlbumIndex = 1

I'd like to point out that using standalone thumbor is not particularly fast - which is why I personally abandoned it in the first place.

I hope this helps?

[arahayrabedian]

oh, a thumbor config - I spun up thumbor real quick using docker, but yeah, I could provide a guide on what I used. Gimme a few more mins.

[arahayrabedian]

So personally I'm a huge fan of docker for prototyping things - this is not what I'd use on prod but it should get you started towards figuring out your dev thumbor and extrapolating to set up your prod thumbor.

I use this nice docker-compose setup, using this configuration example

with that you can get to something like:

version: '2'
services:
  thumbor:
    image: apsl/thumbor:latest
    #build: thumbor
    volumes:
      - data:/data
      - logs:/logs
    ports:
      - "8081:8000" # thumbor port
    environment:
      - DETECTORS=['thumbor.detectors.feature_detector','thumbor.detectors.face_detector']
      - SECURITY_KEY=secret  # again, make sure this matches up with 50mm
      - AWS_ACCESS_KEY_ID=<your_aws_key>
      - AWS_SECRET_ACCESS_KEY=<your_aws_secret>
      # Is needed for buckets that demand the new signing algorithm (v4)
      # - S3_USE_SIGV4=true
      - TC_AWS_REGION=eu-west-1
      - TC_AWS_STORAGE_BUCKET=<bucket_that_matches_50mm's>
      - TC_AWS_STORAGE_ROOT_PATH=  # intentionally left blank
      - TC_AWS_RESULT_STORAGE_BUCKET=thumbor-docker
      - TC_AWS_RESULT_STORAGE_ROOT_PATH=result_storage
      - STORAGE=tc_aws.storages.s3_storage
      - UPLOAD_PHOTO_STORAGE=tc_aws.storages.s3_storage
      # Two options for result storage
      - RESULT_STORAGE=thumbor.result_storages.file_storage
      # and if you want to share the result storage cache between docker-images:
      # - RESULT_STORAGE=tc_aws.result_storages.s3_storage
      # it works without share it, but consider any docker-image will download
      # and process the result_image
    networks:
      - app
volumes:
  data:
    driver: local
  logs:
    driver: local
networks:
  app:
    driver: bridge

again, I've smacked this config up in a few minutes without much thought - so there will be very rough edges, but it should get you started/on your way toward figuring out the S3 bucket backend and/or how to get thumbor to work locally with 50mm for dev purposes

Of course - you have to make sure your buckets, secrets, and ports match up.

To summarize, 50mm reads your bucket, generates a thumbor link for your images when you request a gallery, your browser makes the request to that link/image which then causes thumbor to look in to the same S3 bucket, download the photo using a signed URL (photo is assumed to be private on S3, remember!), potentially process it, and then serve it.

I hope that all makes sense?

If you'd still be interested in thumbor's http method of talking to S3 do say so, I can't implement it now but I'd be happy to tackle it eventually.

[kylemarsh]

Nope! This sounds great. I'll play around with this a bit and see if I have other problems. Thanks so much

On Feb 20, 2018 13:04, "Ara Hayrabedian" notifications@github.com wrote:

So personally I'm a huge fan of docker for prototyping things - this is not what I'd use on prod but it should get you started towards figuring out your dev thumbor and extrapolating to set up your prod thumbor.

I use this nice docker-compose setup https://github.com/APSL/docker-thumbor, using this configuration example https://github.com/APSL/docker-thumbor/blob/master/configuration_examples/docker-compose/aws-s3-storage.yml

with that you can get to something like:

version: '2'services: thumbor: image: apsl/thumbor:latest

build: thumbor

volumes:
  - data:/data
  - logs:/logs
ports:
  - "8081:8000" # thumbor port
environment:
  - DETECTORS=['thumbor.detectors.feature_detector','thumbor.detectors.face_detector']
  - SECURITY_KEY=secret  # again, make sure this matches up with 50mm
  - AWS_ACCESS_KEY_ID=<your_aws_key>
  - AWS_SECRET_ACCESS_KEY=<your_aws_secret>
  # Is needed for buckets that demand the new signing algorithm (v4)
  # - S3_USE_SIGV4=true
  - TC_AWS_REGION=eu-west-1
  - TC_AWS_STORAGE_BUCKET=<bucket_that_matches_50mm's>
  - TC_AWS_STORAGE_ROOT_PATH=  # intentionally left blank
  - TC_AWS_RESULT_STORAGE_BUCKET=thumbor-docker
  - TC_AWS_RESULT_STORAGE_ROOT_PATH=result_storage
  - STORAGE=tc_aws.storages.s3_storage
  - UPLOAD_PHOTO_STORAGE=tc_aws.storages.s3_storage
  # Two options for result storage
  - RESULT_STORAGE=thumbor.result_storages.file_storage
  # and if you want to share the result storage cache between docker-images:
  # - RESULT_STORAGE=tc_aws.result_storages.s3_storage
  # it works without share it, but consider any docker-image will download
  # and process the result_image
networks:
  - appvolumes:

data: driver: local logs: driver: localnetworks: app: driver: bridge

again, I've smacked this config up in a few minutes without much thought - so there will be very rough edges, but it should get you started/on your way toward figuring out the S3 bucket backend and/or how to get thumbor to work locally with 50mm for dev purposes

Of course - you have to make sure your buckets, secrets, and ports match up.

To summarize, 50mm reads your bucket, generates a thumbor link for your images when you request a gallery, your browser makes the request to that link/image which then causes thumbor to look in to the same S3 bucket, download the photo using a signed URL (photo is assumed to be private on S3, remember!), potentially process it, and then serve it.

I hope that all makes sense?

If you'd still be interested in thumbor's http method of talking to S3 do say so, I can't implement it now but I'd be happy to tackle it eventually.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/agile-leaf/50mm/issues/19#issuecomment-367118336, or mute the thread https://github.com/notifications/unsubscribe-auth/AARoBm5U7nDOM-Ye8jdz-Z8XwlM3Gtxlks5tWzMRgaJpZM4SMdmH .