firecow / gitlab-ci-local

Tired of pushing to test your .gitlab-ci.yml?
MIT License
2.03k stars 115 forks source link

"Invalid gitlab-ci configuration! It have failed the json schema validation." though Gitlab pipeline runs fine #1269

Closed dfineSvenFlock closed 1 week ago

dfineSvenFlock commented 1 week ago

Hi, since the latest update received via homebrew my Gitlab yml files do not work with gitlab-ci-local any longer. The files itself are fine as the pipelines in Gitlab do not complain about invalid yaml.

parsing and downloads finished in 694 ms
Invalid gitlab-ci configuration! It have failed the json schema validation. Dump the following to the pipeline editor to debug: stages:

Here is the complete rendered output. As Gitlab itself does not raise any issues, I have no idea what's the problem. I'd assume this is a bug?

Minimal .gitlab-ci.yml illustrating the issue As the output is very long and I cannot point to the issue it is best to make a checkout of the repo (link below) and run gitlab-ci-local.

Expected behavior gitlab-ci-local --file .gitlab-ci.yml --list should render the Gitlab YAML files properly and present the jobs The original source code for the main file can be found at https://gitlab.com/eurodat/trustee-platform/-/blob/main/.gitlab-ci.yml?ref_type=heads

Host information macOS gitlab-ci-local 4.51.0 (installed via Homebrew)

Containerd binary Docker Desktop for Mac (latest version)

ANGkeith commented 1 week ago

seemed like your manifest has some config that does not satisfy the json schema.
for eg. image

Kudos to @hverlin, once https://github.com/firecow/gitlab-ci-local/pull/1266 is merged in, we should have a nicer error message ie: image

ANGkeith commented 1 week ago

As Gitlab itself does not raise any issues, I have no idea what's the problem. I'd assume this is a bug?

It's not really a bug, it's just that gitlab.com do not do any schema validation on the backend https://github.com/firecow/gitlab-ci-local/pull/1248#issuecomment-2146971764

dfineSvenFlock commented 1 week ago

Hi, thanks for your quick reply.

Besides using the code from MR to get a detailed output of the invalid schema, is there a way to do it otherwise? I tried yamlint both on command line and in IntelliJ. But besides the first error you printed, everything else is ok. I also tried the CIlint from Gitlab showing me no errors.

image
hverlin commented 1 week ago

You need to use the pipeline editor (see https://docs.gitlab.com/ee/ci/pipeline_editor/)

dfineSvenFlock commented 1 week ago

@hverlin : can you help me out with the error message on your PR? I says both that it requires to be a string and in the second item, it tells me it should be an array. Can this be both true?

Removing the paths: [] from the artifacts definition was enough to get GitLab-Ci-Local back to working. Could that be false positives?

PigeonF commented 1 week ago

The issue seems to be your CI configuration: all your *:sonarcloud jobs extend the .sonarcloud:template job as the last entry in extends, which has artifacts:paths: [].

Due to the way arrays are merged in GitLab CI this means that all your *:sonarcloud jobs end up with artifacts:paths: [] (which the schema says is invalid, since the schema requires at least one entry).

The solution should be to remove the artifacts:paths block from .sonarcloud:template.

The same applies to artifacts:reports:junit in the argo-client job.

Using the following patch, I am able to run gitlab-ci-local --list in your repo.

diff --git a/cluster/pipeline/broker.gitlab-ci.yml b/cluster/pipeline/broker.gitlab-ci.yml
index d4f269349d7c1e55..46845b433bba8dc8 100644
--- a/cluster/pipeline/broker.gitlab-ci.yml
+++ b/cluster/pipeline/broker.gitlab-ci.yml
@@ -27,8 +27,6 @@ argo-client:
     paths:
       - "$CI_PROJECT_DIR/.m2"
   artifacts:
-    reports:
-      junit: []
     paths:
       - "$CI_PROJECT_DIR/.m2"
     expire_in: 1 day
diff --git a/cluster/pipeline/common.gitlab-ci.yml b/cluster/pipeline/common.gitlab-ci.yml
index e7f31c0bdfe1bc73..fd78bbd944cc952e 100644
--- a/cluster/pipeline/common.gitlab-ci.yml
+++ b/cluster/pipeline/common.gitlab-ci.yml
@@ -90,7 +90,6 @@
         exit 1
       fi
   artifacts:
-    paths: []
     expire_in: 1 day

 .get-images:
dfineSvenFlock commented 1 week ago

@PigeonF Thanks. The error in junit confused me because another hidden job had a valid junit reports value. I will close this ticket and I will be so happy once the PR from @hverlin is in main :-).