Improved yaml input conversion and generated json schema

[anders-kiaer]

webviz-config today supports the following:

• Automatic conversion of user input (from yaml file) to correct type (based on type hinted plugin __init__). Also reports early if input argument is of type wrong (non-castable). https://github.com/equinor/webviz-config/blob/76aeb0102d7a7395a9af83f3935b28561c508564/webviz_config/_config_parser.py#L88-L107
• Generating a json/yaml schema based on type hinted plugin __init__ function. See this video for an example of todays functionality. https://github.com/equinor/webviz-config/blob/76aeb0102d7a7395a9af83f3935b28561c508564/webviz_config/_docs/_create_schema.py#L56-L71

Current solution however has some limitations/drawbacks:

• We need to maintain code on our side, which we might be able to out source to pydantic.
• We don't support nested structures (or very limited). E.g. config can't check that a List[str] is actually a list of strings (today it only supports list - and checking that it is a list....).
• Overloaded init functions are not supported (both wrt. input type conversion, and yaml/json schema. Supporting this could significantly improve e.g. yaml schema user experience for complex plugins in webviz-subsurface (if you give one argument, the editor could immediately fill in other required arguments matching that function signature).
• ...along the same route is supporting TypedDict as argument type hint (early check that keys are correct/present). Complex plugins sometimes typically have arguments which are dictionaries. Giving them the possibility to use TypedDict would improve the user experience.

Related issues:

• 199

• 358

Additional information:

• Snippet that can take a function signature and both convert yaml/dict input to correct types, as well as json schema from the function signature, all using pydantic available here: https://github.com/samuelcolvin/pydantic/issues/2127#issuecomment-728285603.
• As stated in @typing.overload documentation, only the last function signature is by default available during runtime. This gives some challenges, as we probably don't want to introduce some specific @overload Webviz decorator (such that plugin authors can use mypy natively, as well as not introduce a new concept/feature to learn). However, the Python docs can be interpreted as it is not by default available during runtime⭑

⭑ Since webviz docs and webviz schema are user entrypoints, we are in control of import order of different packages. The following could be used in those subcommands to monkeypatch overload to do what we want. Prototype code:

class SomePlugin(WebvizPluginAbc):
    @typing.overload
    def __init__(response: None) -> None:
        ...
    @typing.overload
    def __init__(response: int) -> typing.Tuple[int, str]:
        ...
    @typing.overload
    def __init__(response: bytes) -> str:
        ...
    def __init__(response):

import inspect
import importlib
import typing
from webviz_config import WebvizPluginABC

overloaded_functions = []
def new_overload_func(func):
    overloaded_functions.append(func)

typing.overload = new_overload_func

from webviz_config import plugins

def check_overloaded_functions():
    for func in overloaded_functions:
        class_reference = getattr(inspect.getmodule(func), func.__qualname__.rsplit(".", 1)[0])
        if inspect.isclass(class_reference) and issubclass(class_reference, WebvizPluginABC):
            argspec = inspect.getfullargspec(func)
            print(argspec.annotations)
            print(class_reference)

check_overloaded_functions()

will give as output

{'return': None, 'response': None}
<class '[...].SomePlugin'>
{'return': typing.Tuple[int, str], 'response': <class 'int'>}
<class '[...].SomePlugin'>
{'return': <class 'str'>, 'response': <class 'bytes'>}
<class '[...].SomePlugin'>

[anders-kiaer]

Moving this to prioritized as it is important in order to prepare for #504.