search

nanostudio-org / nanogallery2

a modern photo / video gallery and lightbox [JS library]
https://nanogallery2.nanostudio.org
754 stars 110 forks source link

S3 as a source #100

Open rrzz opened 6 years ago

rrzz commented 6 years ago

I could help design an S3 as a source feature, let me know if you're interested and I will put some work into it

Kris-B commented 6 years ago

Hi, yes, I'm interested! A server side application may be required, doesn't it?

wykananda commented 6 years ago

Thumbnails. As far as I know, S3 can't automatically generate thumbnails on upload or download. However the AWS Lambda service can be set up to generate thumbnails on upload and perhaps download too (though I've never tried to do that and I think it wouldn't be the best approach).

What I did for my needs was to put together a Lambda function that (a) generates and saves a thumbnail whenever I upload an image to the S3 bucket and then (b) copies that uploaded image file to my free Flickr account for use with Nanogallery.
This gives me a backup on S3 and the image manipulation and organization powers of Flickr.

Anyhow, if rrzz adds S3 support to Nanogallery, I think it could only need to require 3 steps from the end-user:

  1. Create an Images bucket and an Image-thumbnails bucket on S3 and configure them for Static Website Hosting -> https://docs.aws.amazon.com/AmazonS3/latest/user-guide/static-website-hosting.html [Very Easy]

  2. Install and Configure the Lambda function on AWS that will trigger and generate a thumbnail whenever a file is uploaded to that Images bucket and place it in the Images-Thumbnail bucket. [Advanced] but a nice step by step tutorial could be created to easily walk users through it.

  3. Set the Static Website URLs of the Images and Image-thumbnails in the Nanogallery object. [Very Easy]

S3 has no concept of folders but instead uses filename prefixes to simulate a directory-like structure (ex. "dirA\dirB\photo1.jpg"). So, the S3 code in nanogallery could parse these prefixes to organize the files into Albums etc.

I'm happy to share my thumbnail code with you, rrzz, if you like. I think you'll find quite a few examples available of how to do that out there on the web as well. I think there may even be a pre-canned example in the Lambda service itself.

S3 support would be a great addition to Nanogallery for sure. I think it would be one of the first, if not the first, to provide this feature.

rrzz commented 6 years ago

WOW great input and you've had more time to think of this than I have.

This is exactly along the lines of what I was thinking along with using AWS Cognito via AWS Javascript SDK for the browser to use any of the AWS apis in support of more features than should be possible without a server. For brevity, we can create IAM roles in the AWS accounts with fine grained public access to an aws account and have the app making calls right from the browser.

We could use SQS, SNS, DynamoDB, ElasticSearch Service etc for cache, state, queue, notification etc. I have been looking for a gallery project like this for a while and haven't seen comprehensive attempts. I would love to help architect these as I make time and hear feedback.

Any input or service ---> gallery/app feature ideas would be great if you're reading this.

How should we develop these ideas wykananda?

wykananda commented 6 years ago

Hi rrzz. I'm afraid I'm not in a position to offer so much more to this noble endeavor at this time - though I'll certainly be keeping an eye on it.

Reflecting on your last post and thinking a bit more on it...

So, as it stands, we can already use S3 with Nanogallery by simply provisioning an S3 bucket as a Static website and utilizing Nanogallery2's self-hosted-images abilities. Yay!!

And then for caching/better performance etc., we can create a CloudFront origin in AWS for the S3 bucket.

Without any additions to Nanogallery, we're already up and running using S3. Not too shabby really but no automated album creation - so a lot of manual labor.

Adding the auto-thumbnail creation using AWS Lambda removes one tedious step.

Hmm, how to automate album creation like Nanogallery does for Flickr and Google Photos...

Yep, as you mention, using AWS Cognito via AWS Javascript SDK for the browser seems like a neat approach. For example, this could be used to return and generate the "items" content for the Nanogallery object parsing out all the needed album structure and data from the S3 object names and/or metatags. This approach requires no changes to Nanogallery itself - which is pretty cool.

...but then this is where a DynamoDB table perhaps should be considered for storing the metadata as you mention?

Still, I'm wondering if this might be a good first step to automating the whole affair... Implementing a Lambda function to generate thumbnails and meta/tags whenever a file is uploaded and using the AWS Javascript SDK to parse that out into a Nanogallery-friendly "items" object.

Alternatively, we could skip the AWS Javascript SDK requirement all-together by having that Lambda function update/recreate a JSON file in the S3 bucket on every upload. The JSON file contains the nanogallery friendly "items" object data. This file is then loaded into Nanogallery's items variable via $.getJSON and so on and voila. Perhaps this would make a good Phase 1 approach.

Something like: a) A static-website provisioned s3 bucket optionally turned into a CloudFront origin. b) A Lambda function for generating the thumbnail and Nanogallery "items"-compatible JSON file when a user uploads a file to the bucket. c) Use $.getJSON or somesuch to pull the JSON file into the Nanogallery "items" variable.

If this worked well and we had a decent tutorial, then perhaps Kris would consider incorporating step (c) natively. I'm thinking it might be relatively trivial.

It might be possible to get something like that working in just a day or two of effort. That's not too bad really.

What do you think rrzz?