How to specify complex input parameters or properties in command line options

[andy-maier]

There is a number of create / update command options that are not yet implemented because they need to set parameters/properties that have complex data types, and we do not yet have an agreed-upon way to specify those. There are a few cases of comma-separated string values, but more complex things than that are a gap today.

Example: The "Create Image Activation Profile" operation has an input parameter "ssc-network-info" whose type is an array of "ssc-network" data structures, which are dictionaries with 5 properties, one of which is again a dictionary with 2 properties.

This issue is to discuss and agree on a way to specify the value of such complex parameters or properties in zhmc command line options.

I did some research on this problem, but could not find anything useful. My research was probably not exhaustive, though.

The following lists some options I could think of:

Option 1: JSON

Description:

• The option value is specified as a single-line JSON value.

Pro/Con:

• Pro: Easy to parse using the "json" module.
• Pro: Consistent with specifying JSON in options of ansible-playbook commands.
• Con: JSON is not the most easy way for humans to specify something.
• Con: Requires backslash escaping of JSON double quotes if shell variables are to be used.

Example for "ssc-network-info":

--ssc-network-info '[{"port": 444, "ipaddr-type": "static", "vlan-id": 53, "static-ip-info": {"type": "ipv4", "ip-address": "10.11.12.13"}}]'

Example for "ssc-network-info" when using shell/env variables:

--ssc-network-info "[{\"port\": 444, \"ipaddr-type\": \"static\", \"vlan-id\": $vlan_id, \"static-ip-info\": {\"type\": \"ipv4\", \"ip-address\": \"$ip_address\"}}]"

Option 2: name=value syntax

Description:

• Values of simple types are specified as:
  • none/null: null
  • bool: true or false
  • int: decimal string representaton, parseable by int()
  • float: string representaton that is parseable by float()
  • string: string value, with character range and representation as for JSON string values. See note (1)
• Values of dict/object type are specified using {}, their items use a key=value syntax, with one or more blanks between the items. Alternative: The items are comma-separated instead of blank-separated.
• Values of array/list type are specified using [], their items use a value syntax, with one or more blanks between the items. If the array items are dicts, the items are separated by zero or more blanks. Alternative: The items are comma-separated instead of blank-separated.
• Note (1):
  • The rules for JSON string values are: Representable character range is U+0020 to U+10FFFF plus the whitespace chars (\b, \f, \n, \r, \t). Can optionally be surrounded by double quotes. Double quotes around the value are mandatory if the escaped string value contains blanks or backslash-escapes. When parsing a double quoted (=escaped) string, these backslash-escape sequences are supported: \", \\, \/, \b, \f, \n, \r, \t, \uNNNN, \uNNNNNN. When creating an escaped string (in double quotes), then double quotes, backslashes, the whitespace chars (\b, \f, \n, \r, \t) and (?) any Unicode character outside of 7-bit ASCII are mandatory to be backslash-escaped.
  • Escaped JSON strings can be parsed using the Python "json" module, by building a JSON string that is a dict with one item whose value is the string to be parsed. Command line version to try it out:

    es='"ab\ncd"'
    python -c "import json; es=r'${es}'; x=json.loads('{\"x\":'+es+'}'); s=x['x']; print(repr(s))"

Pro/Con:

• Pro: Easier to specify for humans, compared to JSON
• Con: Requires our own parser
• Pro: No backslash escaping if shell variables are to be used (except if they contain blanks or backslash-escapes)

Example for "ssc-network-info":

--ssc-network-info "[{port=444 ipaddr-type=static vlan-id=53 static-ip-info={type=ipv4 ip-address=10.11.12.13}]"

Example for "ssc-network-info" when using shell/env variables:

--ssc-network-info "[{port=444 ipaddr-type=static vlan-id=$vlan_id static-ip-info={type=ipv4 ip-address=$ip_address}]"

Examples with a (fantasy) escaped string value:

--properties '{description="ab\ncd"}'
--properties "{description=\"ab\\ncd\" name=\"${name}\"}"

Option 2 is implemented in PR #516

[simon-spinner]

Have you tried to use string concatenation in option 1?

--ssc-network-info '[{"port": 444, "ipaddr-type": "static", "vlan-id": '$vlan_id', "static-ip-info": {"type": "ipv4", "ip-address": "'$ip_address'"}}]'

You need to put the shell variables in single quotation marks (basically end and start the string again). But at least bash typically concatenates everything and passes it as a single string to the program

[simon-spinner]

Another observation: Option 2 looks very similar to the single line syntax of YAML. The most obvious difference is that you need commas between fields of an object and use : instead of =

[andy-maier]

Option 3: Single line YAML (aka "Flow Style")

The YAML spec calls this Flow Collection Styles.

Some sources claim that quotes around property names and string values would be needed, but that does not seem to be the case. Example: https://yaml-online-parser.appspot.com/?yaml=prop%3A+%7Bresult%3A+42%2C+name%3A+alice%7D%0A&type=json Also the YAML spec confirms that no quotes are needed, see Example 7.15: https://yaml.org/spec/1.2.2/#rule-ns-s-flow-map-entries

Note that the flow collection style YAML is a superset of JSON, so double quotes around property names and string values can also be used.

Example for "ssc-network-info":

--ssc-network-info "[{port: 444, ipaddr-type: static, vlan-id: 53, static-ip-info: {type: ipv4, ip-address: '10.11.12.13'}}]"

Example for "ssc-network-info" when using shell/env variables:

--ssc-network-info "[{port: 444, ipaddr-type: static, vlan-id: $vlan_id, static-ip-info: {type: ipv4, ip-address: '$ip_address'}}]"

Examples with a (fantasy) escaped string value:

--properties '{description: ab\ncd}'
--properties "{description: ab\\ncd}"

I must say, I like option 3 the best.

We probably need to write a help topic with a short explanation of the syntax, since the YAML spec is hard to read for occasional users.

More understandable descriptions of flow style that we could refrence:

• https://www.yaml.info/learn/flowstyle.html

Probably not so good:

• https://www.tutorialspoint.com/yaml/yaml_flow_styles.htm - shows the optional !!xxx YAML qualifiers
• https://tutorialreference.com/yaml/styles/yaml-flow-style - shows the optional !!xxx YAML qualifiers
• https://dev.to/this-is-learning/yaml-collections-sequences-and-mappings-4meb - not only about flow style

[andy-maier]

I have implemented option 3 in PR #537 - if we agree on that PR, I consider the decision made for using option 3.

[andy-maier]

We agreed on PR #537, so we will continue using option 3 in the future. This issue is now complete.