Store main docstrings in the manifest

[tcompa]

Refs

• https://github.com/fractal-analytics-platform/fractal-common/issues/90
• https://github.com/fractal-analytics-platform/fractal-server/issues/549
• https://github.com/fractal-analytics-platform/fractal/issues/531

[tcompa]

There need to exist new functions/scripts in in the dev subpackage, that implement the addition of docs_link and docs_info fields into each task of the manifest. I propose to follow the same logic that we use for parameters JSON Schemas:

• A function that extracts the appropriate docstring (probably this can be a small modification of something that we already have)
• A new_docs_info.py script (similar to new_args_schemas.py) that loops over all tasks from an existing manifest and updates the docs_info and docs_link entries in the corresponding manifest item. At the end of this script, the manifest is updated (on-disk), and it must be added/committed. This is the way to update the manifest when we update a given docstring or when we want to change docs_link.
• A check_docs_info.py (similar to check_args_schemas.py) which has the same structure but only checks that the current on-disk manifest is up-to-date, rather than overwriting. This can be included in the GitHub action which currently runs check_args_schemas.py, to make sure we do not merge a PR with an out-of-sync manifest.

What should actually be written in the manifest?

link

docs_link: either a link the the docs homepage, or to the specific function. Note: docs are not version-specific, so that they will be out-of-sync in the future. I'd rather include a generic homepage link.

docstring

I'd suggest that docs_info includes only the short/long function descriptions (i.e. the first block of the docstring), and not the parameters' descriptions (which are already stored in the JSON Schema).

[mfranzon]

In principle we can avoid new files creation because we can extend the function create_schema_for_single_task in fractal_tasks_core/dev/lib_args_schemas.py with an auxiliary function that put into the manifest docs_info and docs_link.

Is necessary another check? Does the function _compare_dicts in fractal_tasks_core/dev/check_args_schemas.py already checks for missing keys in task args?

[tcompa]

In principle we can avoid new files creation because we can extend the function create_schema_for_single_task in fractal_tasks_core/dev/lib_args_schemas.py with an auxiliary function that put into the manifest docs_info and docs_link.

Is necessary another check? Does the function _compare_dicts in fractal_tasks_core/dev/check_args_schemas.py already checks for missing keys in task args?

See https://github.com/fractal-analytics-platform/fractal-tasks-core/pull/486#issuecomment-1685779622. The two files are actually needed (or a single file, but with two different features).