Figure out a scalable validation procedure for fields and forms

[jsheunis]

SHACL is meant for validation. It has built-in information that would allow validating graph data. Normally, in the world of linked data / RDF, this would happen via some tool that supports shacl-validation of RDF data. But for the UI that we are building, we want to bring those validations to the front. Ideally, each form field should validate its data on entry. Additionally, there could be a validation step for a whole node when it is saved, likely when the user hits a "Save" button.

Vuetify supports various ways of data validation, and validation error display to users, depending on the type of form entry field. E.g. input types for text fields: https://developer.mozilla.org/en-US/docs/Web/HTML/Element/input. More generally, Vuetify has the rules mechanism: https://vuetifyjs.com/en/components/forms/#rules

An easy starting point will be to look at required fields. Shacl will assign a cardinality constraint of sh:minCount 1 if a node property is required, and that can translate easily into a required form field.

On the form level, the validation process could include some javascript-based graph validator. Should look further into https://github.com/rdf-ext/shacl-engine

Lastly, the agent/service that ingests the collected data could also decide to run some server-sode validation. Perhaps pyshacl could be useful for that.

[jsheunis]

Interim notes:

• the rules mechanism works well on a per-field basis, and can both be programmatically constructed from the SHACL shape's constraints (for general cases, such as a required field or a regex pattern, this is done in the rules.js composable), and rules can also be appended by validation steps that are specific custom editor component.
• For form-level validation, an idea is to assign each field in a form to a ref, which would make them all accessible via refs in the component's methods. A form-level validation step can then do both the built-in validate() as well as loop through all form fields and run separate validation if needed. Something for which this could be useful is to have a custom list of all validation errors to be printed alongside the form.

An example of the second note:

  // Define form fields within a reactive object
    const formRefs = reactive({
      field1: '',
      field2: null,
      field3: [],
    });

  const validateAllFields = () => {
      let valid = true;

      // Get all refs inside the form
      const refs = Object.values(formRefs);

      refs.forEach((fieldRef) => {
        if (fieldRef && fieldRef.validate) {
          const fieldValid = fieldRef.validate();

          if (!fieldValid) {
            valid = false;
            if (fieldRef.errorBucket && fieldRef.errorBucket.length) {
              formErrors.value.push(...fieldRef.errorBucket);
            }
          }
        }
      });

      return valid;
    };

[jsheunis]

The form level validation above that involves iterating through all fields of a form was implemented in 96f933ebfdef0c85c963d1423faceb591651a45b.

The remaining task is in-browser shacl-based validation of a data graph against a set of shapes. This could perhaps be seen as redundant, given that per-field validation is already done on entry (or on the form level when being iterated over) on the basis of the relevant property shape. But on the other hand it could also be seen as a necessary fail safe in case there are constraints in the shapes graph that aren't yet implemented in shacl-vue or that might operate at a higher level than individual fields.

After looking at https://github.com/rdf-ext/shacl-engine, some code to try out:

import rdf from 'rdf-ext';
import SHACLEngine from 'shacl-engine';

async function validateData(dataGraph, shapesGraph) {
  const validator = new SHACLEngine({ factory: rdf });
  const report = await validator.validate(dataGraph, shapesGraph);
  if (report.conforms) {
    console.log('valid data');
  } else {
    console.log('validation failed:');
    for (const result of report.results) {
      console.log(`Focus Node: ${result.focusNode.value}`);
      console.log(`Severity: ${result.severity.value}`);
      console.log(`Message: ${result.message ? result.message.value : 'no message'}`);
    }
  }
}

validateData(dataGraph, shapesGraph)
  .then(() => console.log('Validation complete'))
  .catch((error) => console.error('Validation error:', error));

[jsheunis]

https://github.com/psychoinformatics-de/shacl-vue/commit/dc7e9788409c48d5b286a7a1fe1cf62f11288d6c has an initial prototype of what shacl-engine-based validation could look like. Need to test this with useful data, and also improve the validation error messaging associated with failures.