Open dacieri opened 3 years ago
Thanks @dacieri for reporting this issue...
glab
heavily relies on the GitLab API. I have found two issues on GitLab that are related to this issue:
https://gitlab.com/gitlab-org/gitlab/-/issues/217018
https://gitlab.com/gitlab-org/gitlab/-/issues/31971
I believe this issue should be resolved for repos hosted on GitLab.com and self hosted instances running v13.6+.
Is your project hosted on GitLab.com or a self-hosted instance?
My host is gitlab.cern.ch, we should be running GitLab Enterprise Edition 13.7.6-ee
I think this is an issue with the GitLab CI Lint API: https://gitlab.com/gitlab-org/gitlab/-/issues/320778
The existing API endpoints allow for linting of CI scripts that only include scripts on the default branch, but not newly added ones.
Can you share a few insensitive part of your CI yaml?
sure, that's the .gitlab-ci.yml
include:
- project: 'hog/Hog'
file: '/hog.yml'
ref: 'Hog2021.1-1'
- local: '/.before_script.yml'
- local: "/.user_jobs.yml"
- local: '/.devel_to_master.yml'
#---------------------------------------------
# HW
#---------------------------------------------
# - local: '/Top/profpga_duo_test/gitlab-ci.yml'
# - local: '/Top/profpga_duo_csf_plus_pt.0/gitlab-ci.yml'
# - local: '/Top/profpga_duo_pt.0/gitlab-ci.yml'
# - local: '/Top/hal/gitlab-ci.yml'
# - local: '/Top/l0mdt/gitlab-ci.yml'
#---------------------------------------------
# Modules
#---------------------------------------------
- local: '/Top/csf.0/gitlab-ci.yml'
- local: '/Top/csf.1/gitlab-ci.yml'
- local: '/Top/ptc_mpi/gitlab-ci.yml'
- local: '/Top/lsf/gitlab-ci.yml'
- local: '/Top/upt/gitlab-ci.yml'
- local: '/Top/ucm/gitlab-ci.yml'
- local: '/Top/hp/gitlab-ci.yml'
- local: '/Top/heg/gitlab-ci.yml'
- local: '/Top/hps_csf/gitlab-ci.yml'
- local: '/Top/hps_lsf/gitlab-ci.yml'
# - local: '/Top/daq/gitlab-ci.yml'
- local: '/Top/mpl/gitlab-ci.yml'
# - local: '/Top/tdc/gitlab-ci.yml'
# - local: '/Top/mtc/gitlab-ci.yml'
#---------------------------------------------
# Builds
#---------------------------------------------
# - local: '/Top/ult/gitlab-ci.yml'
- local: '/Top/ult_tp_dev/gitlab-ci.yml'
- local: '/Top/ult_tp_ucm/gitlab-ci.yml'
- local: '/Top/ult_tp_csf/gitlab-ci.yml'
- local: '/Top/ult_tp_csf_ptmpi/gitlab-ci.yml'
- local: '/Top/ult_tp_lsf/gitlab-ci.yml'
# - local: '/Top/ult_tp_lsf_questa/gitlab-ci.yml'
# - local: '/Top/ult_tp_lsf_xsim/gitlab-ci.yml'
# - local: '/Top/ult_daq/gitlab-ci.yml'
# - local: '/Top/csf_plus_pt/gitlab-ci.yml'
we make an extensive use of the include feature, so I don't know if that would impact. The main yaml that is included is this https://gitlab.cern.ch/hog/Hog/-/blob/master/hog.yml, which includes other three yml in the reference repository. These yamls include both template jobs that are later defined in the locally included yml (i.e. - local: '/Top/csf.0/gitlab-ci.yml'
) and normal jobs that are executed by the CI (https://gitlab.cern.ch/hog/Hog/-/blob/master/YAML/hog-main.yml).
Maybe the project is too complex? :)
Thanks a lot @dacieri... I finally figured this is an issue from our part. We use the general CI validation endpoint, which doesn’t have any namespace specific context so it sees the local include as belonging to a different unknown project.
We should use the Project-specific CI validation endpoint, which is quite young (i.e. introduced in GitLab v13.6), but it uses all namespace specific data available, including variables, local includes, and so on.
I will submit a PR with a fix. This means we would have to compromise users on self-hosted GitLab instances using versions older than v13.6
What if we just hit both endpoints at the same time? We can do that concurrently. If the Project endpoint just returns a 404, we can just ignore that, and fallback to the general CI validation endpoint
What if we just hit both endpoints at the same time? We can do that concurrently. If the Project endpoint just returns a 404, we can just ignore that, and fallback to the general CI validation endpoint
Nice one... I think we should go with that
This issue has been automatically marked as stale because it has not had recent activity. We haven't had the time to address it yet, but we want to keep it open. This message is just a reminder for us to help triage issues.
I looked into this a bit - I tried using the Validate a CI YAML configuration with a namespace API, which seems to work, but it has an issue where includes are resolved from the project's default branch, and you don't seem to be able to override that :frowning_face:
This issue has been automatically marked as stale because it has not had recent activity. We haven't had the time to address it yet, but we want to keep it open. This message is just a reminder for us to help triage issues.
Dear all, I have just discovered this interesting project, which looks very promising to me. I was trying the CI linter option on one of my project, and I get this invalid error:
Naturally the yml is correct, but maybe the linter has some problems with the included ymls? In my case, I include the CI configuration first from another gitlab project, where all stages are defined, and then in my local files I extend the pre-defined stages. If I paste the yml on the gitlab online CI linter, I don't get an unvalid response, in comparison.