type to represent options dictionary

[ueno]

I stumbled on this when reading #19. It seems to be inevitable that crypto API provides an abstraction of optional parameters, because some algorithms take algorithm specific parameters, such as degree of parallelism in Argon2. In #19, it is represented as a handle type (options) with access functions (set, set_u64, get, get_u64). I wonder if this can be defined in a more generic way, such as an array of name-union pairs:

;;; Type of optional attributes.
(typename $optiontype
  (enum u8
    $signed
    $unsigned
    $float
    $bytearray
  )
)

(typename $option_bytearray
  (struct
    (field $buf (@witx const_pointer u8))
    (field $buf_len $size)
  )
)

;;; The option value.
(typename $option_u
  (union $optiontype
    (field $signed i64)
    (field $unsigned u64)
    (field $float f64)
    (field $bytearray $option_bytearray)
  )
)

;;; The content of the `options` element.
(typename $option
  (struct
    (field $name string)
    (field $value $option_u)
  )
)

;;; The array of options.
(typename $options (array $option))

This is analogous to a{sv} data type in D-Bus, which means an array of string-variant entries ({sv}), where v can represent any type in a type-safe way.

[jedisct1]

For bindings authors, typed unions may be the most painful non-native type to deal with.

My concern is that when writing these bindings, constructing and deconstructing a (name, typed union) array is going to be slower and more complicated than just calling a function specialized for a type.

Do we need option types beyond unsigned integers and byte arrays?

There may be some but I can't think of any example where negative values or floats would be needed.

[ueno]

For bindings authors, typed unions may be the most painful non-native type to deal with.

Yes, I realized that while I'm experimenting with the interface :-) However, it has a good property: the callers don't need to maintain the lifetime of options object by themselves. If it is backed by a handle, we need to provide a options_close function that shall be called after the use, or a special semantics such as to transfer the ownership to the operation (like glib's floating reference).

My concern is that when writing these bindings, constructing and deconstructing a (name, typed union) array is going to be slower and more complicated than just calling a function specialized for a type.

If it was implemented naively, that would be. I was wondering if there could be a support for this kind of structure at the Wasm/WASI level as it is pretty generic. @sunfishcode do you have any suggestions in this regard?

[jedisct1]

The options handle can be automatically closed when the state is closed. But reusing the same options to, e.g. encrypt multiple messages is likely to be very common.

Being able to keep an options handle around is convenient, and from the host perspective, options only need to be parsed once. If we pass an array, the host has to parse it every time.

Also, if the options object is constructed, an error can be returned by the set() call when trying to set an invalid or unsupported global option. If we pass an array, we won't get much information besides EINVAL for the global array.

[jedisct1]

a options_close function that shall be called after the use

Good point.

This is not specitic to this submodule, or even to WASI, but bindings in languages without finalizers/destructors such as JavaScript indeed need to explicitly close handles, and some of these closing calls can easily be forgotten.

What could really help debug these issues for anything using WASI is the ability to report the current number of opened handles, so that leaks can be easily spotted. Possibly with more details such as the handle type or what function created it.

Not sure that it should be part of any APIs, but at least having this in some implementations can be extremely useful when writing JavaScript bindings.

[ueno]

It seems that this is the area that interface-types could help. As we don't have it currently, the handle approach might be the best workaround so far.

[sunfishcode]

I agree; it sounds like a handle is a good approach for now, and we can discuss other options as interface types and other options become available.

[ueno]

OK; let's close this for now (maybe good to add a comment around the definition of options and similar types). Thanks for the input!