usnistgov / dioptra

Test Software for the Characterization of AI Technologies
https://pages.nist.gov/dioptra/
Other
220 stars 33 forks source link

Add workflow for entrypoint yaml validation #615

Open keithmanville opened 2 months ago

keithmanville commented 2 months ago

Add a new workflow for validating the task_graph portion of the entrypoint yaml

This should make use of the existing validation functions here: https://github.com/usnistgov/dioptra/blob/main/src/dioptra/task_engine/validation.py

The endpoint should look something like: POST /workflows/entrypointValidate { "task_graph": "contents", "plugin_ids" [...], "entrypoint_parameters": { .. } } With the response being a message stating the either the yaml is valid or not (with the reason provided by the validation function).

We want to be able to validate an entrypoint yaml before an Entrypoint Resource has been created by the user. To accomplish this the user will need to provide some details of the entrypoint.

See here for an example of constructing the full task engine yaml: https://github.com/usnistgov/dioptra/blob/03d295ea1d433bf67e6dc89f0fbd9e006abed9de/src/dioptra/restapi/v1/workflows/lib/export_task_engine_yaml.py#L67

Additionally, the entrypoint services should be updated to prevent the creation of snapshots with invalid yaml. They should return an appropriate error message containing the validation errors from the validation function.

Definition of Done: