Fvk diagram testing

[fvankrieken]

382

First pass at generative diagrams for mapping source input to output. Focus here is more on given a list of source datasets, generating a map that shows to some extent where they come from and what steps occur.

A couple changes to actual data library code as a part of this

• adding path as a field to all template files with script as data source where it's relevant rather than these paths just being hard-coded in the python scripts themselves, so that the info is more mine-able
• adding a scriptor interface with a standard way of getting version and path when ingesting a dataset via script

Very open to feedback on this. Could definitely continue to make improvements down the road in this area as well.

For the graphs, there's just the script within data_flow_diagrams that can be called (after invoking poetry shell). Right now it just generates a couple diagrams, doesn't take arguments. Didn't optimize for cli arguments because this is not an operational script.

[fvankrieken]

LGTM! is it worth using ScriptorInterface in at least all other .py scripts that use the @property ... pattern? from a text search I'm seeing some like dcas_ipis.py

So actually that was one that I missed, thanks for the catch