feat+refactor(engine): Validate trigger input + dsl module refactoring

[daryllimyt]

Features

• Add trigger input validation
  • We use entrypoint.expects to specify a schema
  • We currently only support

    VALIDATION_TYPES = {
    "duration": timedelta,
    "datetime": datetime,
    "str": str,
    "int": int,
    "float": float,
    "bool": bool,
    # List and dict types are WIP
    }

• We now validate trigger inputs on incoming webhooks to fail fast without scheduling workflows
• When we receive an incoming webhook:
  1. Validate the webhook request/authenticity
  2. Get the workflow definition DSLInput
  3. Then perform validation using the DSLInput and the payload
  4. On failure, return a 400 bad request with the details
  5. Otherwise, continue as usual
• Add /validate-workflow endpoint
• Add tracecat dev validate cli command to hit validation endpoint
  • Pass --all to scan through all workflow yamls. Currently only checks playbook dir
  • Pass --data to send a trigger input to be validated against entrypoint.expects

Changes

• Break dsl.common into smaller modules (to combat circular imports)
  • io: fsspec file related
  • validation: SchemaValidatorFactory + trigger validation related
  • models: Shared data models + types
• Attempt to delete underlying Temporal schedule despite db Schedule not found

Todos

• Option to control verbosity

  Testing

• AWS GD slack works
• all unit tests passing

Example

Given a workflow definition YAML(tests/data/workflows/unit_error_fatal.yml):

...
entrypoint:
  ref: a
  # Expect a json object
  expects:
    start_time: datetime
    end_time: datetime
    duration: duration
...

Successful

Trigger input in .dev/json

{
    "start_time": "2024-05-01T00:00:00Z",
    "end_time": "2024-07-01T12:00:00Z",
    "duration": "PT10S"
}

Outputs:

❯ tracecat dev validate -f tests/data/workflows/unit_error_fatal.yml --data @.dev/test.json
Validating unit_error_fatal.yml
{'status': 'success', 'message': 'Workflow passed validation'}

Error

Trigger input in .dev/json

{
    "start_time": "aaaaaa",
    "end_time": 1,
    "invalid": "PT10S"
}

Output:

❯ tracecat dev validate -f /Users/daryllim/dev/org/tracecat/tests/data/workflows/unit_error_fatal.yml --data @.dev/test.json
Validating unit_error_fatal.yml
Failed to validate workflow!
{
  "status": "failure",
  "message": "Trigger input validation error",
  "errors": [
    {
      "ok": false,
      "message": "Validation error in trigger inputs (fatal_error_workflow__trigger_input_validator). Please refer to the schema for more details.",
      "detail": {
        "detail": {
          "errors": [
            {
              "type": "datetime_from_date_parsing",
              "loc": [
                "start_time"
              ],
              "msg": "Input should be a valid datetime or date, input is too short",
              "input": "aaaaaa",
              "ctx": {
                "error": "input is too short"
              },
              "url": "https://errors.pydantic.dev/2.6/v/datetime_from_date_parsing"
            },
            {
              "type": "missing",
              "loc": [
                "duration"
              ],
              "msg": "Field required",
              "input": {
                "start_time": "aaaaaa",
                "end_time": 1,
                "invalid": "PT10S"
              },
              "url": "https://errors.pydantic.dev/2.6/v/missing"
            },
            {
              "type": "extra_forbidden",
              "loc": [
                "invalid"
              ],
              "msg": "Extra inputs are not permitted",
              "input": "PT10S",
              "url": "https://errors.pydantic.dev/2.6/v/extra_forbidden"
            }
          ],
          "schema": {
            "additionalProperties": false,
            "properties": {
              "start_time": {
                "format": "date-time",
                "title": "Start Time",
                "type": "string"
              },
              "end_time": {
                "format": "date-time",
                "title": "End Time",
                "type": "string"
              },
              "duration": {
                "format": "duration",
                "title": "Duration",
                "type": "string"
              }
            },
            "required": [
              "start_time",
              "end_time",
              "duration"
            ],
            "title": "fatal_error_workflow__trigger_input_validator",
            "type": "object"
          }
        }
      }
    }
  ],
  "metadata": {
    "filename": "unit_error_fatal.yml"
  }
}