Design: deconflate `<unset>` into `<unset>` and `<null>`

[aqw]

Currently, onyo get returns <unset> for both a) keys that are missing and b) keys that exist but have an empty value.

This is also the case when querying (--match), making it impossible to differentiate the two states.

This limitation also becomes apparent when deciding upon behavior of stacked templates (#703). A more explicit syntax would remove the ambiguity.

The issue is naming. <unset> can be read as both active (unset this key) or passive (I am an unset value). On the positive side, the active form is the exact same verb as onyo unset. For that reason, I propose:

• <unset>: the key does not exist
• <null>: the key exists, but there is no value

The name <null> feel intuitive, as null is a valid keyword in YAML. We usually use an empty value, but others could use the keyword. And just for trivia, because nothing is ever simple in YAML, the following all mean null:

- null  # or Null or NULL
- ~     # yes, a tilde means null...
-       # empty

The use cases I see are:

1) onyo get output (the values in the output columns) 2) onyo get --match a='<unset>' b='<null>' 3) onyo set --keys a='<unset>' b='<null>' 4) onyo new templates (and thus also subsequent templates, keys, and templates)

These keys accompany <list> and <dict> (which only declare type, not content; [] and {} declare empty content).

None of these type-tags shall be written to an asset (as they have no meaning to YAML). They are allowed to be present in a template, because templates (and TSVs) are just alternate forms of passing data to onyo new --keys, and they provide meaningful instructions to onyo new.

Open Questions: 1) I'm still not thrilled about the name <unset>. My next best are <absent> and <missing>, neither of which feel all that much better. 2) There is an additional type-tag discussed in #703: <inherit>. This would leave existing content untouched, but create an empty key if missing. This would be of little to no use for onyo set, as onyo get can already filter for key/value states. So the only advantage is for onyo new. However... everything in onyo new acts like a waterfall. <inherit> would allow that waterfall to go upstream. So I am skeptical. Plus, the next idea would be "create a key with X value if missing." Which there is no clear syntax for. In general, the more I think about this the more I am against this. But I wanted to raise it.

[bpoldrack]

Agree on the aim here.

Re naming:

• I kinda get the trouble with <unset> , but I can't think of anything better.
• I like <null> better than the <empty> suggested in #703.

The second point, however, raises another open question: What about an explicitly declared empty string (key: "")? Currently, we do match it with the implemented notion of <unset>. Should this match <null>? Or do we treat it like any other value, since it technically is a value? I would lean towards the latter, since it quickly becomes messy otherwise: Matches <null>, but needs to be written as ""? What value would get print? Empty string or <null>? How would that be consistent with actual null?

As for <inherit>: I'm not in favor of this. From my POV, this is exactly the point:

everything in onyo new acts like a waterfall. <inherit> would allow that waterfall to go upstream.

I think, a template that is meant to be inherited from, shouldn't specify a value that's more specific than a "derived" asset. So, if you're in the situation where <inherit> is needed, I think that means the templates you're using simply aren't made for that.

[aqw]

The second point, however, raises another open question: What about an explicitly declared empty string (key: "")

I think the right thing is to error out because it's ambiguous. But we error with a helpful suggestion, specifically explaining that if they want <null>, they should specify <null>. If they want an empty string, they need to quote it correctly: "''".

We disallow passing an empty value.

Note, this exposes a use-case for <empty>: match an empty string, dict, or list. But I don't have strong emotions about that.

do we treat it like any other value, since it technically is a value? I would lean towards the latter, since it quickly becomes messy otherwise:

I completely agree. We print it out as "". Less magical; more explicit.

if you're in the situation where <inherit> is needed, I think that means the templates you're using simply aren't made for that.

I am also strongly trending in this direction. I think it's complexity without a clear need yet. We can always add it later if we find that this is a crucial use-case. But from where I stand right now, the benefits are not obvious, while making things much more complex.