Design JSON API

[lilith]

In designing the JSON API, we want to preserve evolvability at all points. This will mean that several aspects will be versioned, and we will endeavor for early versions to be highly forward-compatible.

We expect to expose more than one style of JSON API. We'll start by designing the lowest-level format. Higher level formats should be easy to compose from the lowest-level format. For example, a 'linear' mode can create a graph from a simple list of operations.

Let's start with a few knowns about our initial internal design:

1. Operation graphs which deal in single bitmap frames are easy to understand, optimize, and reason about (compared to operation graphs which would deal with multi-frame inputs, outputs, and multi-frame codec state). This does not isolate the operation graph from receiving information (like bitness, palette, codec kind, encoding quality, rotation metadata) from both decoders and encoders, but does constrain it from issuing more than a single 'writeframe' command to any encoder. This has worked well in practice so far.
2. By isolating the operation graph to have a set of single-frame inputs and outputs, we must place the burden of responsibility for dealing with frame/page selection elsewhere. Let's call the unit responsible for this the 'builder'. So far, we have not implemented anything past a single-frame builder and a gif disassembler. A complete builder would both read and produce GIF and/or WebP animations.
3. We also abstract I/O, so that decoders and encoders are not bound to implement too many interfaces. All I/O objects should be created before beginning any work on a job.

Thus, the highest-level involved parties in executing a task would be:

• I/O objects, with numeric IDs
• A builder, which orchestrates opening I/O, beginning decoding, and applying the operation graph in its own way to those I/O objects, and dealing with additional metadata.
• The operation graph, with numeric IDs for input/output frames
• Configuration for the builder, including debugging parameters

With expectation of how the parties may evolve, we can select a versioning system that seems to cause the lowest friction.

We would expect versioning at:

1. The JSON task container schema itself, when our assumptions become invalid.
2. The selection of a particular version of a builder permits sub-sections to be interpreted differently.
3. As we expect all builders to re-use the same schema for individual nodes/operations, we expect to version the 'operation set'. Internally, this may map to backwards-compatible parsers, but the internal graph and implementation of these nodes will also be versioned - but for plugin compatibility instead. If significant operational differences are required, then we may have to map version ranges from the operation set API to specific internal API versions. Exposing the internal graph version information only makes sense in the context of plugins, which we haven't yet explored in depth.

Additional needs and complications arrive when this API is exposed over HTTP instead of via a command-line tool or FFI invocation. We aim for re-use and consistency, so this design will attempt to address all needs.

[shmuelie]

Just thought I link to the OGC's Web Map Service specification which happens to have similar needs/issues as us http://www.opengeospatial.org/standards/wms

[lilith]

After skimming the document I don't see much overlap. Perhaps you could open an issue and describe the intersection you see?

On Aug 24, 2016 12:18 PM, "Shmueli Englard" notifications@github.com wrote:

Just thought I link to the OGC's Web Map Service specification which happens to have similar needs/issues as us http://www.opengeospatial.org/ standards/wms

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/imazen/imageflow/issues/75#issuecomment-242160416, or mute the thread https://github.com/notifications/unsubscribe-auth/AAGln3xvR1wGYoAh2N-yyjGbn2-97Wryks5qjIrcgaJpZM4JsSKC .

[shmuelie]

Overlap I see:

1. Designed to be transport agnostic while also having specifications for transport over HTTP
2. Version handling

[lilith]

Ah, I assumed there was something image-related you were referring to. I didn't see any references to JSON. Perhaps there are more widespread conventions around JSON API versioning and transport agnosticism we can link in here?

[lilith]

I drafted up a sample job for a common frame of reference. It's a bit verbose, but nearly all bits are optional. Next I need to figure out what a result should look like.

{ 
  "jobSchemaVersion": "0.1",
  "builder":"prototype",
  "builderVersion": "0.1",
  "builderVersionComment": "Is there benefit to separating builder and builderVersion?",
  "builderConfig": {
    "enable_jpeg_block_scaling": "true",
    "processAllGifFrames": "true",

    "debug": {
      "record_graph_iterations": "false",
      "record_frame_images": "false",
      "render_last_graph": "false",
      "render_graph_iterations": "false",
      "render_animated_graph": "false"
    }
  },
  "io": {
    "0": {
      "direction": "input", 
      "url": "http://s3-us-west-2.amazonaws.com/imageflow-resources/test_inputs/roof_test_800x600.jpg",
      "checksum": {
        "alg": "djb2",
        "hash_hex" : "8ff8ec7a8539a2d5"
      }
    },
    "1": {
      "direction": "output", 
      "system": "stdout",
      "comment": "stdout/stdin only permitted with imageflow_tool"
    },
    "2": {
      "direction": "input",
      "placeholder": "0",
      "comment": "with imageflow_tool, input/output resources could be specified at the command line"
    },
    "3": {
      "direction": "output",
      "file" : "output.png",
      "comment": "writing to the local filesystem only permitted with imageflow_tool"
    },
    "4": {
      "direction": "input",
      "bytes_hex": "89504E470D0A1A0A0000000D49484452000000010000000108060000001F15C4890000000A49444154789C63000100000500010D0A2DB40000000049454E44AE426082",
      "comment": "inline data not suggestion for large files"
    }
  },
  "perFrame": {
    "graph": {
      "nodeSetVersion":"1.0",
      "nodes": {
        "0": {
          "type": "decode",
          "io_id": 0
        },
        "1": {
          "type": "scale",
          "width": 400,
          "height": 300,
          "fit": "max"
        },
        "2": {
          "type": "encode",
          "codec": "encode_png",
          "io_id": 3
        }
      },
      "edges": [
        {"from": 0, "to": 1, "type": "input"},
        {"from": 1, "to": 2, "type": "input"}
      ]
    },
    "stepsComment": "this and 'graph' would be mutually exclusive",
    "steps": [
        {
          "type": "scale",
          "width": 400,
          "height": 300,
          "fit": "max"
        },
        {
          "type": "encoder",
          "codec": "encode_png"
        }
      ]
    }
  }
}

[lilith]

For lists of items with unique numeric IDs, is {"0": {}, "1": {} } or [{"id":0}, {"id":1}] preferred?

[shmuelie]

I prefer the second [{"id":0},{"id":1}]

[lilith]

I've found that many (de)serializers enforce the determinism of schema enforcement based on key name (with the exception of the keys of perfectly free-form maps, like used for having IDs as keys bellow). I find myself not ideologically opposed to this, as it is highly consistent and forward-flexible.

This usually means an extra layer of curly braces, I.e,

1

{
        "nodes": {
            "0": {"decode": { "ioId": 1 } },
            "1": {"rotate90" : null}

        },
        "edges": [
            {"from": 0, "to": 1, "kind": "input"}
        ]
    }

Or, if one is interested in favoring polymorphism over composition,

2

{
        "nodes": [
            {"Decode": {"id": 0, "io_id": 1 } },
            {"Rotate90" : {"id":1}
        ],
        "edges": [
            {"from": 0, "to": 1, "kind": "Input"}
        ]
    }

or extra depth, maintaining composition but still using an array

3

{
        "nodes": [
            {"id": 0, "node": {"Decode": { "io_id": 1 } } },
            {"id": 1, "node": {"Rotate90" : null } }
        ],
        "edges": [
            {"from": 0, "to": 1, "kind": "Input"}
        ]
    }

Preferences between these?

[lilith]

Going with Sample 1.

Tracking development in #88

If there is feedback, it will probably happen later, and more issues can be opened.