Open keighrim opened 6 months ago
All we need is two special characters to build a notation for the standardized string syntax that can be parsed as a dict in python. However, our interfaces (URL or shell CLI) pose lots of constraints on choosing those two. My proposal is at the bottom, but very open to more suggestions. Especially I want to hear input from @owencking as he's our primary user.
Here's an early implementation (by @snewman-aa with some help from @kelleyl IIRC) without a standardized handling of such a parameter;
Specifically, the query
parameter is used to pass a prompt string (for an LLM) and a short label string to save the results in the output MMIF. For example, a user can ask "images with one or two person closed up in the middle and have dense text overlay in the bottom third of the image" and annotate all the "found" images as a chyron
-labeled TimePoint
.
Notation-wise, this implementation is using@
to delimit the prompt string and the label string, then taking advantage of multivalued=True
configuration of the parameter to allow two or more prompt-label pairs. Hence in practice, the URL for asking for chyron
annotation (the example above) will be ;
$ curl --some --options \
"http:localhost:xxxx?query=images%20with%20one%20or%20two%20person%20closed%20up%20in%20the%20middle%20and%20have%20dense%20text%20overlay%20in%20the%20bottom%20third%20of%20the%20image@chyron"
Or since the long string is a part of the URL query string, we can use +
to escape spaces (for a little better readability, lucky us!).
$ curl --some --options \
"http:localhost:xxxx?query=images+with+one+or+two+person+closed+up+in+the+middle+and+have+dense+text+overlay+in+the+bottom+third+of+the+image@chyron"
And finally, when a user wants multiple labeling at the same time, for example,
chyron
credits
$ curl --some --options \
"http:localhost:xxxx?query=images+with+total+black+of+dark+monotone+background+with+two+or+more+lines+of+horizontal+text+evenly+distributed+vertically@credits&query=images+with+one+or+two+person+closed+up+in+the+middle+and+have+dense+text+overlay+in+the+bottom+third+of+the+image@chyron"
Some lessons from this early implementation;
&
however, that character has a very specific meaning in bash
(and many other unix shells). So we must assume that the whole URL string is quoted almost all the time (mostly with double quotes, to give some scripting freedom to users who are more fluent with shell scripting)?
- #
) we can use +
, but it is still users' responsibility to construct and pass a proper URL string when they are sending requests. So I did some research on possible options, and here is some additional information to consider
%hh
form): https://www.rfc-editor.org/rfc/rfc1738.html#section-2.2
# unsafe chars
< > # % { } | \ ^ ~ [ ] `
query
part of a URL is actually allowed with lots of special characters (but many of them are "unsafe" so needs to be escaped anyway): https://www.rfc-editor.org/rfc/rfc3986.html#section-3.4
# (excerpt) CFG rules for query string
sub-delims = "!" / "$" / "&" / "'" / "(" / ")" / "*" / "+" / "," / ";" / "="
unreserved = ALPHA / DIGIT / "-" / "." / "_" / "~"
pchar = unreserved / pct-encoded / sub-delims / ":" / "@"
query = *( pchar / "/" / "?" )
Given all that, my train of thought:
:
as k-v delimiter, and ,
as delimiter between k-v pairs. Fortunately, both are NOT unsafe characters in URL(yay!). However, many SWT annotation labels already contain :
character, so if we want to parameterize "post-binning" of the SWT app under this standardized map-type parameterization, we need to handle the "legacy" keys with colons. And then for ,
characters, I think they are very likely to be used in natural language-based keys or values. So I want to avoid using any common natural language punctuations. mutlivalued=True
can be an elegant solution to eliminate the necessity for one delimiter character (k-v pair delimiter). Downside of it is that the user needs to repeat param=
part in the query string. If the parameter name is long enough, that can be not only tedious annoyance, but also could pose some technical issues in terms of string length. But I still think the benefit outweighs, considering all the (could be totally imaginary) confusions around the concept of properly constructing the parameter value before sending a request. I think it's easier to make users understand by saying that "if you have two or more k-v pairs to pass, call this parameter multiple times" instead of "if you have two or more k-v pairs to pass, use this particular character to "join" all of them to create well-formed URL to send a request".So finally, I'm proposing
:
to delimit between a key and a value:
:
is not a unsafe URL character, and has no special meaning other than the hostname:port
delimiter:
is so universally perceived as k-v delimiter in the primary languages we use (python and json)multivalued=True
for all type="map"
parameters, and let users send multiple k-v pairs calling it repetitively. And some fictional examples of SWT "postbin" parameterization under the proposal
# note colons in subtype labels are simply "collapsed" (S:H >> SH)
$ clams source SOME_INPUT_SPEC | curl -d@- "http:localhost:5001?minFrameCount=5&labelbin=B:bars&labelbin=S:slated&labelbin=SH:slate&labelbin=SC:slate&labelbin=SD:slate&labelbin=SG:slate&labelbin=I:chyron&labelbin=N:chyron&labelbin=Y:chyron&labelbin=C:credit&labelbin=R:credits"
# and of course, users aren't required to pass this long url all the time, since we will have reasonable default values for the `labelbin` param.
# `labelbin` flag can take `?` number of arguments (https://docs.python.org/3/library/argparse.html#nargs)
$ clams source SOME_INPUT_SPEC | python cli.py --minFrameCount 5 --labenbin S:slate SH:slate SC:slate SD:slate SG:slate I:chyron N:chyron Y:chyron C:credit R:credits
First, I agree 100% about using :
to delimit keys and values. If that entails not supporting :
in things like SWT labels, then we probably shouldn't include :
within labels!
(In general, I favor restricting label names to concatenations of "unreserved" characters listed above (from RFC 3986, sec 2.3), with the possible exceptions of ~
and .
. But that's a question for another time.)
As for the way to pass multiple parameters. Yes, I think doing it as multivalued makes sense. (If it were possible to pass a string that could be immediately parsed as JSON or a Python dict, then that would be nice. However, as a practical matter, we'll need to put quotes around the strings before we can use them. So, some pre-processing will be required.) The multivalued solution is elegant.
One additional wrinkle -- maybe a difficult one. We might need even more structure. For parameterizing binning, we might need to associate a key with a set of values, for example to express this kind of structure:
'post': {
'bars': ['B'],
'slate': ['S', 'S:H', 'S:C', 'S:D', 'S:G'],
'opening': ['W', 'O', 'M'],
'chyron': ['I', 'N', 'Y'],
'credits':['C', 'R']
}
I think ,
would be a very natural separator for multiple values. However, Keigh, you said
And then for , characters, I think they are very likely to be used in natural language-based keys or values. So I want to avoid using any common natural language punctuations.
Yes, I can imagine NLP apps where it'd be nice to use a comma within the value of some parameter. But (1) we are already giving up one common punctuation mark with :
, and (2) maybe those sorts of natural language string value parameters make the most sense for top-level arguments, not within these structured "map" type arguments. So, maybe it would be okay to use ,
to delimit parts of these mappy things.
Regarding the nested list parameters, for the time being I think confining all possible data types on the SDK level might not work on scale. We can still keep a minimum confinement of syntax and parser for string-to-string map within the SDK, and leave more complex data passing to app developers. For example, with the example above, the app developer can add additional parameter specification (well-documented in the app metadata ideally) that can handle comma separated strings for a list-type value. Concretely, users can use a URL like this
http://...?post=bars:B&post=slate:S,SH,SC,SD,SG&post=opening:W,O,M&post=chyron=I,N,Y&post=credits=C,R
and then the SDK will pass this dict to app code;
{
"post": {
"bars": "B",
"slate": "S,SH,SC,SD,SG",
"opening": "W,O,M",
"chyron": "I,N,Y",
"credits": "C,R"
},
... # more params
}
Then app developer can add extra process to parse the string values into lists.
post_final = {}
for k, v in params['post']:
post_final[k] = v.split(',')
In short, defining only string-to-string mapping at the SDK doesn't block app developers to implement their own more complex data types.
@keighrim is it possible to have the CLAMS apps receive both runtime params and MMIF in the same object? Something like...
{
"parameters": {
"post": {
"bars": ["B"],
"slate": [ "S", "SH" ,"SC", "SD", "SG" ],
"opening": [ "W", "O", "M" ],
"chyron": [ "I", "N", "Y" ],
"credits": ["C", "R"]
},
"speed": "88 mph",
"gigawatts": 1.21,
"fluxCapacitor": true
},
"MMIF" : {
/* all the glorious MMIF */
}
}
We actually had something very similar with that for parameter passing method in our previous project (LAPPSgrid), but we changed the way in CLAMS since adding that "wrapper" json around MMIF will almost certainly requires users to use some additional piece of software to generate that non-MMIF json at the runtime. For example, with the current (pure) MMIF-in, MMIF-out method, and given the apps are running as (micro) web services here and there, users can simply chain-call them to create a simple CLAMS pipeline
cat EXISTING_MMIF.mmif | curl -d@- -s http://app1:8080 | curl -d@- -s "http://localhost:8001?p1=v1" | curl -d@- -d "http://remote-app3:5555?param=value&more_param=more_value" > final_output.mmif
But if the apps take a MMIF wrapper JSON, users might have to do something like this;
cat EXISTING_MMIF.mmif | wrap_mmif --flag-for-empty-param | curl -d@- -s http://app1:8080 | wrap_mmif --p1-flag v1-value | curl -d@- -s http://localhost:8001 | wrap_mmif --flags for --many params | curl -d@- -d http://remote-app3:5555 > final_output.mmif
And in the LAPPS project, one of "utilities" that our team provided for users was the wrapper tool, but we realized it added not only a fair amount of maintenance cost for us, but also additional discipline and confusion to users.
In fact, the utility for that "wrapping" weren't even a single CLI app but a stack of java APIs, groovy-based DSL, and precompiled interpreter binary, so users had to write actual a code file instead of a shell script of chained curl
s, to allow direct encoding of complex, fully json-compatible data types (also all LAPPS webservices weren't pure HTTP, but all SOAP).
That being said, we can maybe provide an additional route option to users for that wrapped non-MMIF input. That would requires
parameters
And I believe that should be a separate issue for future development.
I created a new issue on the topic of passing parameters as a JSON object (instead of string maps), as it should be discussed in a broader context, not just for map
-typed params.
One more thing I'd like to address before closing this issue: in the current implementation, :
character being used as delimiter blocked users and developers from using the same character inside a label. However we can still implement the parameter parser to be quotation-sensitive, so that users can pass "enclosed" strings with colon characters in them, just like how commas are handled in the CSV format.
For example, right now with SWT mapping, one can't use the original "sub"-labels
# this doesn't work!
map=S:slate,map=S:H:slate,map=S:C:slate,map=S:D:slate,map=S:G:slate
But we can upgrade the parameter parser to allow usage of colons when the values are properly quoted,
# these should be allowed
map=S:slate,map=\"S:H\":slate,map=\"S:C\":slate,map=\"S:D\":slate,map=\"S:G\":slate
map=S:slate,map=\'S:H\':slate,map=\'S:C\':slate,map=\'S:D\':slate,map=\'S:G\':slate
Still remain to decide;
New Feature Summary
We have seen a use case of passing a nested key-value pairs as a single (string) value to a runtime parameter. And then, we are currently working on a generalized stitcher implementation in the SDK (https://github.com/clamsproject/mmif-python/issues/265), that will most likely to require some parameterized label "remapping" dict.
The string values for these parameters would then be internally parsed into hashmap/dict data structures to be used for document processing. So this issue is to discuss provisioning of SDK-level APIs to support such map-typed, string-valued runtime parameter with somewhat predictable common behaviors.
Related
No response
Alternatives
No response
Additional context
No response