Open rufuspollock opened 10 years ago
Almost certainly going to clash, might be easier to somehow specify you want it cached and then get given a URL that is a base 62 encoded int of some description, or a UUID or something?
Hmm, but isn't this just like bit.ly - sure there can be clashes but that's life ...
I think I'd rather have a URL that had a UUID in it than use it on a site and have someone 'accidentally' overwrite it with a picture of kittens.
@rossjones you wouldn't be able to overwrite - like bit.ly this would be write once ...
@simong what do you reckon? how easy is this to implement do you think?
It could work on a first-come first-served basis, similar to dns names?
We'd need to work out how to store the actual files and stick a small database behind the app. I don't think it'll be terribly hard. We just need to figure out a way to easily do this on both okfn as dev. I'll try to come up with a prototype in the next week(s).
@simong agree re first-come, first-served (let's do the simplest thing possible). I suggest we use s3 as the backend for the present (no need for a proper database) plus we need to store images ...
Example:
url: https://openspending.org
name: openspending.jpg
The file would be accessible at
https://webshot.okfnlabs.org/f/openspending.jpg
which would actually be stored at:
var hash = md5('openspending.org'); // ex: f1ab4edaec312de12c...
S3 Url: https://s3.amazon.com/okfn-webshot/f1/ab/4e/da/openspending.org
(The hashing is in place so we don't end up with thousands of files in a single folder) If the file doesn't exist S3 will return a 404, for which we can setup a custom 404 page.
If we would ever be worried about taking up too much space we could make use of S3 their object expiration [1].
WDYT?
[1] http://docs.aws.amazon.com/AmazonS3/latest/dev/ObjectExpiration.html
@simong sorry to miss this at the time. Proposal looks really good. If you need s3 space we can get s3 bucket setup for you.
Small suggestions:
suggest we have an app config option to set the bucket and config option for any root path (so we could locate data in s3 bucket at
s3.amazon.com/okfn-webshot/my/base/path/...
Also implementation detail: suggest
GET webshot.okfnlabs.org/f/openspending.jpg
redirect to s3 bucket rather than proxying to save on b/w and hassle
Would be awesome to get this implemented!
@simong I was just using webshot again (I use it a lot!) and really missed this feature. I think this would be super useful to have.
Following my last comment my only thoughts were:
@simong any time to work further on this - I use this service all the time and I would love to have this ability to save feature.
I'm also wondering if we really care about namespace issues about allowing users to login with e.g. github - it is so easy to implement (but that may be overkill!)
Hi all, I finally had some time available to move this ahead. Currently my s3-store
branch has the following:
/api/store
endpoint that allows you to make a POST
request with all the same parameters as the /api/generate
endpoint with the addition of a name
parameter. That parameter allows you to retrieve the image later at /f/<name>
./api/store
we check S3 for conflicts. If the name
is already used, we respond with a 400./f/<name>
redirect to a signed S3 URL, so the app isn't serving anythingSome questions:
@simong some HTTP nitpicking: if you let your client choose the name, you should use PUT. If your server can choose the name, you should use POST.
@simong first this is fantastic - great work :-) Thoughts
@pietercolpaert personally I'm not too zealous on PUT vs POST - i know that if we are being nicely RESTful we would do it but happy not to be perfect ;-)
It feels like you'd lose some of the value of providing a custom name if we're prefixing timestamps/random strings? Each url would look like /f/mything-Xfe
or /f/2015/09/03/mything
. It isn't something you'd be able to easily remember so you'd have to copy/paste it everywhere. Maybe that's OK though?
@simong i think people will usually need to copy and paste anyway but what's nice is you can just see the url and know what the thing is which is really nice. I'm thinking looking at it that the date one is probably the nicer one.
@simong any update here?
/cc @danfowler something for the next newsletter and generally to advertise!
@simong @rgrp Reserving a place for this in the upcoming newsletter :smile:
@simong do you need anything from me?
@simong any update here - this would be so awesome.
Couple of final thoughts:
.png
to the end of file names
Allow users to specify a short name to save their screenshot at (a bit like bit.ly but for screenshots). E.g. you could do ?filename=... and then you could get that screenshot forever at:
webshot-service/f/{filename}.png