Support loading of GitLab `!reference` data during validation

[simon-liebehenschel]

Preface

GitLab !reference support was added in https://github.com/python-jsonschema/check-jsonschema/issues/112 but there are still some edge cases.

How I verified that there are no duplicate issues in the issue tracker

I searched for the "reference" word between open and closed issues and I found only closed https://github.com/python-jsonschema/check-jsonschema/issues/112.

.pre-commit-config.yaml

  - repo: https://github.com/python-jsonschema/check-jsonschema
    rev: 0.23.1
    hooks:
      - id: check-gitlab-ci
        args: ["--data-transform", "gitlab-ci"]

Valid .gitlab-ci.yml

include:
  - project: "myproject/mynamespace/backend/repositoryname"
    ref: v1.1.22
    file: ".gitlab-ci-common.yml"

test:
  stage: test
  services:
    - name: mongo:6.0.6
      command: ["/bin/sh", "-c", "mongod --logpath /dev/null --bind_ip_all --replSet 'rs0' "]
    - !reference [.common_services_for_test_job, services]
  before_script:
    - echo "Hello"

check-gitlab-ci output

Validate GitLab CI config................................................
Failed
- hook id: check-gitlab-ci
- exit code: 1
Schema validation errors were encountered.
  .gitlab-ci.yml::$.test.services[1]: ['.common_services_for_test_job', 'services'] is not valid under any of the given schemas
  Underlying errors caused this.
  Best Match:
    $.test.services[1]: ['.common_services_for_test_job', 'services'] is not of type 'string'

Expected result

Accept the mentioned YAML syntax.

[sirosen]

Thanks for the detailed report!

Looking back at #112 and looking at the implementation, it looks like the current implementation of !reference is just a passthrough which requires the YAML node to be a list.

I think this request will require implementing actual !reference loading, which will also require an implementation of the include directive.

include requires, in turn, a body of features around cloning a referenced GitLab repo, which makes the scope of this feature nontrivial. One notable caveat is that this cannot work in an environment like pre-commit.ci in which the network is intentionally locked down. There are a good number of other considerations about how to handle this, so I'm not sure how prepared I am to work on this as a feature right now.

I'm wondering if there is perhaps a simpler solution which would get you unblocked in the near term. Perhaps if some capability were added to substitute data in for a reference from a local file? This has come up in other contexts, so if it would be useful to you to be able to stub something in for the !reference node, I can cross-reference what notes and info I have and consider planning something in the near term.

---

As a brief aside, you can omit the args from your hook declaration:

       - id: check-gitlab-ci
-        args: ["--data-transform", "gitlab-ci"]

The check-gitlab-ci hook automatically includes the transform. But there's no harm in specifying it twice.