Building a RESTful API for enabling use of image sequencer on a remote server

[tech4GT]

Some Image processing steps can be resource intensive and it might be difficult for users with low powered devices to run them locally one the device, for this purpose we can setup an Express server with an api that can take an image in request and return the step applied image in response. For this purpose we can use https://www.npmjs.com/package/express for building the server and imgur for hosting the images. The image at all steps will be uploaded to imgur. https://imgur.com We can start very basic with support for a single step and later add functionalities add multiple steps and replace steps The demo will have a choice if user want to run on local machine or remote server for each step, we can also limit this in the basis of user's computer specification to prevent overloading the server

[jywarren]

GitMate.io thinks the contributors most likely able to help are @ccpandhare, and @jywarren.

Possibly related issues are https://github.com/publiclab/image-sequencer/issues/173 (Building out a contributor community for Image Sequencer), https://github.com/publiclab/image-sequencer/issues/23 (Planning Issue for the Image Sequencer GSoC Project), and https://github.com/publiclab/image-sequencer/issues/99 (Make the API Documentation using Swagger).

[tech4GT]

@jywarren please have a look, this is pretty much what i had in mind, for summer maybe we can just start with support for single step in the server, it would be relatively simpler project and would take 1-2 weeks to implement?

[ccpandhare]

@tech4GT a very good idea indeed! But we need to ensure that only those steps which will take more time are routed to the server. We don't want the user to be waiting! And how will we decide which is faster? Local or server (based on internet speed, user configuration, etc.?)

I think this will take more than a week or two... Including testing, bugfixes and improvements. Usually this takes more time than the development itself but it is very important! Maybe you can come up with a timeline?

On Fri, Mar 9, 2018, 22:38 Varun Gupta notifications@github.com wrote:

@jywarren https://github.com/jywarren please have a look, this is pretty much what i had in mind, for summer maybe we can just start with support for single step in the server, it would be relatively simpler project and would take 1-2 weeks to implement?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/198#issuecomment-371876868, or mute the thread https://github.com/notifications/unsubscribe-auth/AT0xnPLA8ua2ryOtUvmq1Af89lPheeU8ks5tcrcBgaJpZM4SkkrG .

[tech4GT]

@ccpandhare actually i was thinking we can have the user decide if he wants to use local device or server but i get it, this can take a little more time now working on the timeline, thanks👍

[tech4GT]

@jywarren @ccpandhare would this be a priority or #190 for summer coz @ccpandhare you are right i cannot do both in the given timeframe, maybe we can work on one after the summer, so should i keep this one on #190 in my proposal? thanks

[jywarren]

I think it'd be really useful for example to implement a completely separate service-based MapKnitter back-end, which could receive full-size images and distort them remotely for people. https://github.com/publiclab/mapknitter/

But I think #190 and anything that can help onboard new people to grow this team is a higher priority -- i've written a bit about my reasoning for this here -- hope that makes sense!

https://publiclab.org/notes/tech4gt/02-19-2018/gsoc-proposal-image-sequencer-v2-processing-on-steroids#c18878

[jywarren]

Getting back to this one, esp after all the great improvements this last summer.

This could involve setting up a new repository in which we have a containerized/dockerized node version of image-sequencer which can accept commands via REST API calls.

The string syntax makes this MUCH easier, by allowing us to pass an image or url plus an instruction set. An image would be output.

Another thing we could offer is a callback url maybe!

So if we imagined what a POST request payload would look like, it might include:

// Request:

{
  img_src: "...",
  callback: "https://...",
  commandString: "blur,brightness..."
}

// Response:

{
  progress_url: "https://...",
  status: "success"
}

[tech4GT]

@jywarren This looks great! I didn't quite get the purpose for callback though. Also I was thinking of starting this one in my winter break.

[jywarren]

The callback is not necessary for most cases. But it could function like a push request back to an originating server which could replace polling for progress. Just thinking ahead to useful architecture.

On Thu, Oct 18, 2018, 9:52 AM Varun Gupta notifications@github.com wrote:

@jywarren https://github.com/jywarren This looks great! I didn't quite get the purpose for callback though. Also I was thinking of starting this one in my winter break.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/198#issuecomment-430847623, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ2suLEVzmV_R3CJUII7sNyib3Q7fks5ul96ngaJpZM4SkkrG .

[tech4GT]

Hmm, I am thinking rather than doing it this way maybe we can use web sockets for this kind of functionality, what do you think?

[jywarren]

I think websockets could be cool, but a basic API may be easier to implement and integrate -- the client may not be submitting from javascript in any case. So perhaps we could work on a RESTful approach first, but then consider if a websockets interface is a good follow-up! What do you think?

On Wed, Oct 17, 2018 at 9:58 PM Varun Gupta notifications@github.com wrote:

Hmm, I am thinking rather than doing it this way maybe we can use web sockets for this kind of functionality, what do you think?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/198#issuecomment-430848607, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ6HneE8klxkyHB9mnC6E5zVCMTiLks5ul-APgaJpZM4SkkrG .

[jywarren]

This could relate to https://github.com/publiclab/pi-builder/pull/42 where we want to run Image Sequencer on the Raspberry Pi!

[jywarren]

@icarito i'm a little curious about this - do you think we could containerize this library and run a "job" processing a batch of image sequencer steps on a batch of images, in a CI process? Such that we could "submit" a job with URLs of a set of images, and get out the processed output as an artifact?

[Mridul97]

This issue seems interesting. I would love to work on this issue!

[vibhorgupta-gh]

@Mridul97 I was working on this. Would you like to collaborate?

[Mridul97]

Yes, sure!

[vibhorgupta-gh]

@Mridul97 amazing! Give me a couple of hours, I'll update you on whatever I have managed to get done and commit it as well. Not at home right now. Is that fine?

[Mridul97]

Yes. That's fine!

[Mridul97]

@VibhorCodecianGupta I have created a branch building_a_RESTful_API and invited you to collaborate, you can push your commits there and I will push mine as well!

[vibhorgupta-gh]

I just accepted, sweet! Will do by tonight 👍

[jywarren]

this is so cool! One thing we were thinking of here is to build a Dockerfile around this library. I wonder if the RESTFUL API code would best be kept quite separate from the main codebase - like a wrapper around it -- please keep this in mind as you're working on it! Thanks!

On Tue, Jan 1, 2019 at 7:50 AM Vibhor Gupta notifications@github.com wrote:

I just accepted, sweet! Will do by tonight 👍

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/198#issuecomment-450726956, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ9ZttBBLcJZB9Axn2Oat9kaWA6hZks5u-1lbgaJpZM4SkkrG .

[Mridul97]

Yes, I agree it will be better if we keep the RESTFUL API code separate from main codebase!

[vibhorgupta-gh]

@jywarren agreed.

@Mridul97 shall we decide the architecture then? I think initiating a separate repository for this project would be appropriate, as @jywarren suggested a few comments above. That can include both the express app and the dockerized container for image sequencer. Thoughts?

[Mridul97]

It sounds good! So let's start by first creating it for single step and then improve it from there! As suggested above the images will be hosted at imgur so we will return a URL of the modified image! we will take the url of the image and the step to be applied by the user at the time of calling the API and later a set of steps for applying multiple steps and return a set of URLs of modified images after each step!

[vibhorgupta-gh]

Yes, sounds good and viable!

[jywarren]

This is awesome. If you decide on a project name -- maybe 'image-sequencer-app'? Or something else? -- I can create a publiclab repository we can use as the root. Sound good?

On Wed, Jan 2, 2019 at 9:32 AM Vibhor Gupta notifications@github.com wrote:

Yes, sounds good and viable!

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/198#issuecomment-450877594, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ18EAhJmzj06JUNItVKyBmFXhR9Aks5u_MLdgaJpZM4SkkrG .

[vibhorgupta-gh]

Sounds good!

[Mridul97]

'image-sequencer-app' sounds good!

[jywarren]

Great! Also let's keep in mind the ability to run serverless... Would this be possible in the scenario we've planned out?

https://i-programmer.info/news/141-cloud-computing/12405-gitlab-goes-serverless.html

[jywarren]

https://github.com/publiclab/image-sequencer-app

[jywarren]

And just a note that the upload to imgur needn't be implemented in the first version. What's the most standard way for a serverless API to return an image?

[jywarren]

https://serverless.com/

[tech4GT]

This is awesome!! This can also serve as a very good example use case to image sequencer in node.js.

[tech4GT]

@Mridul97 @VibhorCodecianGupta I think you guys can start off with a simple express based server where you get a stringified sequence in the request and you can expand it and run it one the backend. We can then send the final image either as multipart data or the imgur url

[harshkhandeparkar]

In this case the user will not be able to preview the image evry step because of internet transfer speed so the user will have to enter the whole sequence first and then get the end result. This may result in unexpected results and finding the error can be difficult. So can we use a small 24px * 24px image locally to preview each step and then process the main image on the express app.

[tech4GT]

Harsh, what you are talking about is the integration of the API with the demo, I think we can do that later, since we are developing both the projects separately we should focus on the API part first and we can integrate it with the demo later.

[harshkhandeparkar]

That's good. What I actually wanted to ask is how will the user get the preview for intermediate steps even from the API? This will defeat the purpose of image sequencer's non-conventional workflow in which a new image is created every step.

[harshkhandeparkar]

I'm just curious because I love the idea of RESTful APIs.

[jywarren]

I think that for the initial implementation we can return just the final image. But later, we could return a list of URLs for the image at each step.

Will Express be compatible with the emerging "serverless" format? Just checking, 'cause that'd be cool!

On Thu, Jan 3, 2019 at 6:38 AM HarshKhandeparkar notifications@github.com wrote:

I'm just curious and because I love the idea of RESTful APIs.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/198#issuecomment-451119826, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ35DBd38pk5WNzjtZQn6fdc7j-eDks5u_euXgaJpZM4SkkrG .

[tech4GT]

@jywarren have a look at this https://www.npmjs.com/package/serverless-express

[jywarren]

Oohhhhhh!!! 👍🏽

On Sat, Jan 5, 2019, 4:21 AM Varun Gupta <notifications@github.com wrote:

@jywarren https://github.com/jywarren have a look at this https://www.npmjs.com/package/serverless-express

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/publiclab/image-sequencer/issues/198#issuecomment-451640524, or mute the thread https://github.com/notifications/unsubscribe-auth/AABfJ6QekMfWagMgE9rnyBelX8FjebrVks5vAG6MgaJpZM4SkkrG .

[jywarren]

Just collecting resources here... We might be able to use Google cloud to store images, or Amazon s3. I think we may have a preference for google cloud however as we can potentially get a discount.

https://dzone.com/articles/serverless-image-processing-patterns-with-aws

[jywarren]

And this mentions a template docker image called lambci/lambda for Amazon: https://read.acloud.guru/serverless-image-optimization-and-delivery-510b6c311fe5

This last one is pretty good, though AWS centric. But probably serverless could be set up with Google cloud too. It uses express. But it seems to just respond with the finished image because they assume the minor image processing they're doing won't take long enough to require a callback or progress updates.

[jywarren]

https://github.com/awslabs/serverless-application-model/tree/master/examples/apps/image-processing-service

[harshkhandeparkar]

@jywarren how about firebase

[vibhorgupta-gh]

Firebase integration is tailor-made for Android use. It is also pretty slow and unoptimised for the web. I don't think we can get better alternatives than AWS/GCP

[harshkhandeparkar]

Hmm.. then I guess GCP is the best option

[harshkhandeparkar]

Firebase is probably better for like a blog or something

[jywarren]

Just pasting in more documentation here - https://www.thepolyglotdeveloper.com/2017/12/convert-nodejs-restful-api-serverless-openwhisk/

[jywarren]

https://github.com/awslabs/serverless-application-model/tree/master/examples/apps/image-processing-service