aiidateam / plumpy

A python workflows library that supports writing Processes with a well defined set of inputs and outputs that can be strung together.
https://plumpy.readthedocs.io
Other
8 stars 17 forks source link

It is not possible to have a dynamic port namespace with `valid_type=dict` #257

Open sphuber opened 1 year ago

sphuber commented 1 year ago

The following fails in the most recent version, whereas it used to work:

In [1]: ns = PortNamespace()
---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
<ipython-input-1-2b2df46c459f> in <cell line: 1>()
----> 1 ns = PortNamespace()

NameError: name 'PortNamespace' is not defined

In [2]: from plumpy.ports import PortNamespace

In [3]: ns = PortNamespace()

In [4]: ns.valid_type = dict

In [5]: ns.validate({'a': {'b': 1}, 'b': {'test': 'string'}})
Out[5]: plumpy.ports.PortValidationError("Error occurred validating port '.a..b': Invalid type <class 'int'> for dynamic port value: expected <class 'dict'>")

The reason is that if a value is encountered in a port namespace that is dynamic, the code assumes it is another nested namespace, as opposed to a "leaf" value that just happens to have type dict.

There are valid usecases where one would want to define a dynamic namespace that takes any number of dictionaries but that is currently not supported. It would be great if this could also be supported.

It is not clear if this use case can be supported at the same time of supporting arbitrarily nested portnamespaces which is currently the case. It seems impossible for the code to be able to distinguish dictionaries that are intended to be a leaf value or a nested namespace.

Maybe if valid_type=dict is explicitly defined, we can assume the namespace to be "shallow" and every value is considered to be a leaf value.