clamsproject / clams-python

CLAMS SDK for python
http://sdk.clams.ai/
Apache License 2.0
4 stars 1 forks source link

alternative route for passing "pre-parsed" runtime parameters #224

Open keighrim opened 4 months ago

keighrim commented 4 months ago

@keighrim is it possible to have the CLAMS apps receive both runtime params and MMIF in the same object? Something like...


{
    "parameters": {
        "post": {
            "bars": ["B"],
            "slate": [ "S", "SH" ,"SC", "SD", "SG" ],
            "opening": [ "W", "O", "M" ],
            "chyron": [ "I", "N", "Y" ],
            "credits": ["C", "R"]
        },
        "speed": "88 mph",
        "gigawatts": 1.21,
        "fluxCapacitor": true
    },
    "MMIF" : {
        /* all the glorious MMIF */
    }
}

Originally posted by @afred in https://github.com/clamsproject/clams-python/issues/197#issuecomment-2013664342


And my comment at the time;

We actually had something very similar with that for parameter passing method in our previous project (LAPPSgrid), but we changed the way in CLAMS since adding that "wrapper" json around MMIF will almost certainly requires users to use some additional piece of software to generate that non-MMIF json at the runtime. For example, with the current (pure) MMIF-in, MMIF-out method, and given the apps are running as (micro) web services here and there, users can simply chain-call them to create a simple CLAMS pipeline

cat EXISTING_MMIF.mmif | curl -d@- -s http://app1:8080 | curl -d@- -s "http://localhost:8001?p1=v1" | curl -d@- -d "http://remote-app3:5555?param=value&more_param=more_value" > final_output.mmif 

But if the apps take a MMIF wrapper JSON, users might have to do something like this;

cat EXISTING_MMIF.mmif | wrap_mmif --flag-for-empty-param | curl -d@- -s http://app1:8080 | wrap_mmif --p1-flag v1-value | curl -d@- -s http://localhost:8001 | wrap_mmif --flags for --many params | curl -d@- -d http://remote-app3:5555 > final_output.mmif 

And in the LAPPS project, one of "utilities" that our team provided for users was the wrapper tool, but we realized it added not only a fair amount of maintenance cost for us, but also additional discipline and confusion to users. In fact, the utility for that "wrapping" weren't even a single CLI app but a stack of java APIs, groovy-based DSL, and precompiled interpreter binary, so users had to write actual a code file instead of a shell script of chained curls, to allow direct encoding of complex, fully json-compatible data types (also all LAPPS webservices weren't pure HTTP, but all SOAP).


That being said, we can maybe provide an additional route option to users for that wrapped non-MMIF input. That would requires

  • syntax and semantic specification for that wrapper part
  • data presentation specs in the parameters
    • currently, CLAMS apps are written and documented under the assumption that users will only use simple strings to pass these parameters. How parameters are interpreted in internal data structures are somewhat abstracted and happening under the hood.
  • integration (or just compatibility) with CLI-based entry points (#198 )

And I believe that should be a separate issue for future development.