search

imgflo / imgflo-server

HTTP image processing server based on imgflo
48 stars 7 forks source link

API supporting chaining/combining graphs #33

Closed jonnor closed 6 years ago

jonnor commented 9 years ago

Right now with GET /graph API one can only specify one graph. This makes it hard to have "reusable" graphs, each graph must kindof have the possiblity of every combination one would like to use.

One thing possible right now is to pass one request to another as the input URL. This is not very convenient, and also inefficient: 1) in each step we downloaded and upload images 2) intermediate results are stored even if no-one uses them.

One possible API would be similar to GET /graph, but instead of graphname would take a graph specification where each of the nodes are graphs.

This could maybe be a FBP DSL string?

OUTPORT=filter.OUTPUT:OUTPUT
INPORT=normalize.INPUT:INPUT
INPORT=normalize.FOO:FOO
INPORT=filter.BAR:BAR
normalize(NormalizeGraph) OUTPUT -> INPUT filter(MyFilter)

Problem is that it is very verbose for small subgraphs, each property that should be visible must be exported. An alternative dialect would be to do the edges as FBP DSL string

graph="in(NormalizeGraph) OUTPUT -> INPUT out(MyFilter)"

Exported ports if not specified would default to INPUT in and OUTPUT out.

graph="in(NormalizeGraph) -> out(MyFilter)"

And then allow property assignments directly to the internal:

GET /subgraph/graph=$graph&norm.foo="fofo2"&filter.bar="sfe"

A key challenge is determining the cache key. Using the graph specification directly is the most conservative, even changes in whitespace will cause another key. On the other hand this may become wasteful.

Related to #25

jonnor commented 8 years ago

After fixing an S3 upload issue where content-type header was wrong, it now works to have an imgflo URL as an input of another imgflo graph. See chained_requests testcase: https://github.com/jonnor/imgflo-server/blob/master/spec/graphtests.yaml#L170

jonnor commented 6 years ago

Out of scope