Defaulting in Expressions

[Licenser]

Describe the problem you are trying to solve

Defaulting, or the option to ensure that certain fields are present and if not give them a default value.

Today this can be solved with a construct like:

merge default with event;

where default is the struct holding the default values. However, while this may be elegant it is not something that's easy to understand (nor is it fast, but that's a secondary concern).

This also touches on a somewhat related concern: looking up into structs that do not exist.

a.b.c + d.e.f

where b for example does not exist as a key in a.

Describe the solution you'd like

filling a struct with default values

There are a number of possible approaches however they all have their advantages and problems.

patch

default with struct

a new patch keyword default could be used to specify a default structure for a record.

patch event of
  default {"key": "value"}
end

this would effectively be the same as merge default of event end; just in a more verbose but easier to understand syntax.

default with k/v

a default keyword for each value could be used

patch event of
  default "key1" => "value1",
  default "key2" => "value2",
  # ...
end

This would be very clear on what we set but quite verbose, it could be combined with default {...}

a default operator

This is fairly simple, provide a struct as default values and have them fill the event as it passes through.

a default infix operator

# (could clash with default for path)
let event = event default {"key": "value"};
# or (could cause gramer conflicts)
let event = event default merge {"key": "value"};
# or (could cause gramer conflicts)
let event = event merge {"key": "value"};
# or (uses up new keyword)
let event = event ensure {"key": "value"};
# or (uses up new keyword)
let event = event put_if_absent {"key": "value"};

The second example here is since we'll discuss default for path lookups later and the second option might get around issues but create new ones.

default for lookups

sometimes we want to lookup a value in a nested structure and the path doesn't exist.

return null

This is an option but it comes with serious issues, it would equate null with absent that makes it very hard to distinguish if a value is set to null or absent moving the problem from "What do we do when the value isn't set?" to "is the value set?"

default as a keyword

a.b.c default 7
# or (would burn up a new keyword, also somewhat not as clear ensure might mean we ensure it's always 7)
a.b.c ensure 7

would allow being explicit to when we lookup and replace a default value.

an option type

yes please, but this is a lot of work, would extend the type system, introduce error handling and struct functions so there isn't even a proposed syntax for this

a function

struct::get(struct, key, default)

quite hard given how our functions are implemented would require reworking that, also would be problematic with nested keys. Upside we could do this today in tremor script (slow but doable)

Notes This will require an RFC before implementation.

[mfelsche]

My 2 cents for the lookups:

Defaulting only applies to nested paths, that is paths like a.b or a["b"] or a[42].b. And it is the nesting that can fail at each level if the specified nested key or element is not there. This applies to both arrays and records. I would love to have a means to express an optional nesting step, including a default value in case the nesting step fails. From that perspective the best candidates seem to be:

• base["nesting"] default <DEFAULT> but with this having multiple nesting steps is hard: (base[0] default {}).b default {} (this is currently not possible syntax-wise and needs some more work, also it is cumbersome). Maybe we allow the default to apply to any nested path of arbitrary long nesting, so users get the default if anything along that path fails. For more granular control users need to extract nesting by using multiple steps.
• a function like std::path::get(base, "nested", <DEFAULT>) Maybe we could use the second argument to express something like a jsonpath e.g. "a.b.c" as triple nesting. Or have it accept an array of path segments: std::path::get(base, ["nest1", 1, "nest2"], <DEFAULT>).

[mfelsche]

When it comes to filling a struct with default values the approach that appeals to me the most is the patch operation default which works with both single keys and full records. It transports the intent to modify the given record much better than the standalonedefault operators. And we don't add another operator/language construct/additional tool, but work with what we have and keep the complexity of the language low (in terms of number of constructs/operators etc)

[darach]

PR to follow - we discussed this on community chat and explored alternatives with the disposition:

• The default fn alternative is the most useful variant, and does not need an RFC
• Changing the grammar to support default expressions results in LALR disambiguity - which we are avoiding for cost/benefit
• A try/catch/throw mechanism was explored, and is viable - but, needs an RFC and further discussion - it would also make a good mentorship project

[Licenser]

reopening this as only part of this issue was implemented