search

imgflo / imgflo-server

HTTP image processing server based on imgflo
48 stars 7 forks source link

Batch/job API #25

Closed jonnor closed 6 years ago

jonnor commented 9 years ago

Right now imgflo-server only has a direct GET /process API which takes one processing request. For batch operations, this requires the client to drive the whole job, which is inconvenient as completing it all will take a long time. Potentially hours, depending on size of job.

Create a job with many inputs, a status/progress query API for the job, and then be able to gather the outputs after it is all completed. Possibly also allow custom graph/params per processing request. Job can be thought of as a transaction closed around individual processing requests.

jonnor commented 9 years ago

Possibly internally one should treat /process type request as just a special case of the batch job, where the number of requests is 1.

bergie commented 9 years ago

http://www.l1ghtm4n.com/post/53259404576/patterns-for-rest-api-bulk-operations

jonnor commented 9 years ago

One can now POST /graph, which returns urls where request will eventually be put (on success). Not nice when things fail though, as result will just 404 with no info as to what failed

jonnor commented 8 years ago

For this to be useful, might need to support webhooks for notifying on job completion. That way an API consumer can place their request, store related info in the DB, and when the webhook comes in, lookup in the DB and continue their task.

jonnor commented 6 years ago

Out of scope