Axios proxy via cli options

[mamyn0va]

Minimal .gitlab-ci.yml illustrating the issue

include:
  - project: to-be-continuous/golang
    ref: 4.0.0
    file: templates/gitlab-ci-golang.yml
  - project: to-be-continuous/docker
    ref: 3.2.2
    file: /templates/gitlab-ci-docker.yml
  - template: Jobs/SAST.gitlab-ci.yml
  - template: Jobs/Secret-Detection.gitlab-ci.yml
  - template: Jobs/Dependency-Scanning.gitlab-ci.yml
  - template: Jobs/Container-Scanning.gitlab-ci.yml

Expected behavior It works! :)

Host information Manjaro gitlab-ci-local 4.33.1

Additional context The repo is hosted on a private gitlab instance, and it uses many templates.

Remote include could not be fetched https://gitlab.com/gitlab-org/gitlab/-/raw/HEAD/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml AxiosError: Request failed with status code 400

I don't know why gitlab-ci-local tries to fetch some templates from gitlab.com instead of my private gitlab instance. The 400 error may be due to a proxy error (I'm behind a corporate proxy).

[firecow]

@mamyn0va Can you provide the the full gitlab-ci-local ouput

gitlab-ci-local > textformjn.txt

and paste it here.

[mamyn0va]

There is nothing more :

$ gitlab-ci-local
Remote include could not be fetched https://gitlab.com/gitlab-org/gitlab/-/raw/HEAD/lib/gitlab/ci/templates/Jobs/Secret-Detection.gitlab-ci.yml AxiosError: Request failed with status code 400

Thanks!

[firecow]

Nothing about missing remote's or anything ?

[mamyn0va]

No, and it's easily reproducible with this file:

include:
  - template: Jobs/SAST.gitlab-ci.yml

[firecow]

Not that easy apparently :smiley:

[firecow]

Why is it that you think templates shouldn't be fetched from gitlab.com? Isn't this where the template repository is hosted?

[mamyn0va]

I thought that templates were fetched from the same instance as the .gitlab-ci.yml's one.

-------- Message d'origine -------- Le 17 oct. 2022 à 18:05, Mads Jon Nielsen a écrit :

Why is it that you think templates shouldn't be fetched from gitlab.com ?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: @.***>

[firecow]

And what makes you think that?

[mamyn0va]

The fact that other templates are fetched from my own instance.

[mamyn0va]

Do you have any idea why I'm getting 400 errors ? Is that because I'm behind a corporate proxy ? Is it possible to configure a proxy in the tool ?

[firecow]

No template includes have ever been downloaded from your private gitlab instance, local includes will, but never template includes.

Nah, not really... Can you do regular curl against gitlab.com?

[mamyn0va]

Yes, I can do a regular curl to fetch the template from gitlab.com.

[bcouetil]

Hey 😊

From what I see in the doc, it seems to be fetched... from the same instance, in a specific folder : https://docs.gitlab.com/ee/ci/yaml/#includetemplate

[mamyn0va]

Thanks, so the question is : why gitlab-ci-local fetches the template from gitlab.com ?

[firecow]

Hey 😊

From what I see in the doc, it seems to be fetched... from the same instance, in a specific folder : https://docs.gitlab.com/ee/ci/yaml/#includetemplate

The "templates" link points directly to gitlab.com. Where do you see that information?

[firecow]

I just tried replacing gitlab.com with gitlab.firecow.dk in this https://gitlab.com/gitlab-org/gitlab/-/raw/master/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml link, and the file is 404 on my private instance.

He is seeing 400... Which indicates, someone is telling the request is "bad" not not found.

[bcouetil]

It's expected the 404, by default an instance has no template.

This piece of documentation in particular :

Templates are stored in lib/gitlab/ci/templates. Not all templates are designed to be used with include:template, so check template comments before using one.

Uses a relative path, and it must be empty in your instance.

I agree that the 400 error is something else, that we won't tackle together.

[bcouetil]

Unfortunately I can't test all of this, my current clients are on gitlab.com.

[firecow]

I don't get why it must be empty in my instance... If i push an .gitlab-ci.yml with a template include to gitlab.firecow.dk, the pipeline fetches the file directly from gitlab.com

[bcouetil]

Because templates are defined by the admin, and you did not put any in your folder ? And on the OP instance, they are defined by the admins...

Just guessing at this stage, I can't test any of my assumptions.

[firecow]

You think they are manually added to self hosted instances?

I find that unlikely, since my templates are fetched from gitlab.com, when run on gitlab.firecow.dk...

[greenmaid]

I think I can confirm this error is caused by proxy :

---
include:
  - remote: "https://gitlab.com/Orange-OpenSource/lfn/ci_cd/gitlab-ci-templates/-/raw/master/markdown.gitlab-ci.yml"

job:
  image: alpine
  script:
    - echo 'hello'

this works perfectly without proxy but I got Remote include could not be fetched https://gitlab.com/Orange-OpenSource/lfn/ci_cd/gitlab-ci-templates/-/raw/master/markdown.gitlab-ci.yml AxiosError: Request failed with status code 400 as soon I have with a proxy

[mamyn0va]

Hi @greenmaid, I confirm that it works perfectly without proxy with this file:

---
include:
  - template: Jobs/SAST.gitlab-ci.yml

job:
  image: alpine
  script:
    - echo 'hello'

[firecow]

Ok, let's find out how we shall configure axios proxy then...

[greenmaid]

Looking at tcpdump trace, I can say that requests is send via proxy but without CONNECT (as if it was a http URL)

GET https://gitlab.com/Orange-OpenSource/lfn/ci_cd/gitlab-ci-templates/-/raw/master/markdown.gitlab-ci.yml HTTP/1.1
Accept: application/json, text/plain, */*
User-Agent:  axios/0.27.2
Accept-Encoding: gzip, deflate, br
host: gitlab.com
Connection: close

Proxy answers 400

HTTP/1.0 400 Bad Request
Server: BigIP
Connection: close
Content-Length: 6564

<html>
<FONT face="Helvetica"><big><strong></strong></big>
[...]

For me it is exactly this bug at axios side: https://github.com/axios/axios/issues/4531 with open PR: https://github.com/axios/axios/pull/5037

[greenmaid]

Axios + proxy seems historically... complicated 😵

[firecow]

We could switch http client library

[firecow]

I'm gonna come up with some way to specify axios proxy setting via gitlab ci local options. I've renamed the issue.

[mamyn0va]

Hi @firecow, any update on this ?

[firecow]

@mamyn0va Haven't looked seriously into it yet.