gitlab_project_variable module can't connect to Gitlab API

[kayahk]

Summary

I am trying to transfer a string as a variable into a Gitlab project by using the gitlab_project_variable module. However, any combination of the api_url is leading to an error, mostly to 404 not found. I can curl the Gitlab API manually and acces the project and it's variables, the same URL does not work, regardless of the URI path versions I have tried.

So this is the task:

 - name: write kubeconfig to Gitlab variables
      community.general.gitlab_project_variable:
        api_url: "https://gitlab.local.fqdn/api/v4"
        api_token: "{{ lookup('env', 'GITLAB_TOKEN') }}"
        project: "ansible/ansible-role-name"
        purge: false
        vars:
          KUBECONFIG: kubeconfig

As I have mentioned, I also tried to use "https://gitlab.local.fqdn" or "https://gitlab.local.fqdn/api" or "https://gitlab.local.fqdn/api/v4/projects" etc. as the api_url to no avail.

Usually, I am getting this error (see below in Actual Results for verbose output): fatal: [node_name_XY]: FAILED! => {"changed": false, "msg": "Failed to connect to GitLab server: 404: 404 Not Found"}

The error message is different, when calling "https://gitlab.local.fqdn", though:

fatal: [node_name_XY]: FAILED! => {"changed": false, "module_stderr": "Shared connection to node_name_XY closed.\r\n", "module_stdout": "\r\nTraceback (most recent call last):\r\n  File \"/usr/local/lib/python3.8/dist-packages/gitlab/exceptions.py\", line 304, in wrapped_f\r\n    return f(*args, **kwargs)\r\n  File \"/usr/local/lib/python3.8/dist-packages/gitlab/mixins.py\", line 112, in get\r\n    server_data = self.gitlab.http_get(path, **kwargs)\r\n  File \"/usr/local/lib/python3.8/dist-packages/gitlab/client.py\", line 662, in http_get\r\n    result = self.http_request(\r\n  File \"/usr/local/lib/python3.8/dist-packages/gitlab/client.py\", line 628, in http_request\r\n    raise gitlab.exceptions.GitlabHttpError(\r\ngitlab.exceptions.GitlabHttpError: 404: 404 Project Not Found\r\n\r\nThe above exception was the direct cause of the following exception:\r\n\r\nTraceback (most recent call last):\r\n  File \"/home/ansible/.ansible/tmp/ansible-tmp-1628241539.055113-36853-188751157811488/AnsiballZ_gitlab_project_variable.py\", line 100, in <module>\r\n    _ansiballz_main()\r\n  File \"/home/ansible/.ansible/tmp/ansible-tmp-1628241539.055113-36853-188751157811488/AnsiballZ_gitlab_project_variable.py\", line 92, in _ansiballz_main\r\n    invoke_module(zipped_mod, temp_path, ANSIBALLZ_PARAMS)\r\n  File \"/home/ansible/.ansible/tmp/ansible-tmp-1628241539.055113-36853-188751157811488/AnsiballZ_gitlab_project_variable.py\", line 40, in invoke_module\r\n    runpy.run_module(mod_name='ansible_collections.community.general.plugins.modules.gitlab_project_variable', init_globals=dict(_module_fqn='ansible_collections.community.general.plugins.modules.gitlab_project_variable', _modlib_path=modlib_path),\r\n  File \"/usr/lib/python3.8/runpy.py\", line 207, in run_module\r\n    return _run_module_code(code, init_globals, run_name, mod_spec)\r\n  File \"/usr/lib/python3.8/runpy.py\", line 97, in _run_module_code\r\n    _run_code(code, mod_globals, init_globals,\r\n  File \"/usr/lib/python3.8/runpy.py\", line 87, in _run_code\r\n    exec(code, run_globals)\r\n  File \"/tmp/ansible_community.general.gitlab_project_variable_payload_r2ma_9f_/ansible_community.general.gitlab_project_variable_payload.zip/ansible_collections/community/general/plugins/modules/gitlab_project_variable.py\", line 313, in <module>\r\n  File \"/tmp/ansible_community.general.gitlab_project_variable_payload_r2ma_9f_/ansible_community.general.gitlab_project_variable_payload.zip/ansible_collections/community/general/plugins/modules/gitlab_project_variable.py\", line 305, in main\r\n  File \"/tmp/ansible_community.general.gitlab_project_variable_payload_r2ma_9f_/ansible_community.general.gitlab_project_variable_payload.zip/ansible_collections/community/general/plugins/modules/gitlab_project_variable.py\", line 153, in __init__\r\n  File \"/tmp/ansible_community.general.gitlab_project_variable_payload_r2ma_9f_/ansible_community.general.gitlab_project_variable_payload.zip/ansible_collections/community/general/plugins/modules/gitlab_project_variable.py\", line 157, in get_project\r\n  File \"/usr/local/lib/python3.8/dist-packages/gitlab/v4/objects/projects.py\", line 782, in get\r\n    return cast(Project, super().get(id=id, lazy=lazy, **kwargs))\r\n  File \"/usr/local/lib/python3.8/dist-packages/gitlab/exceptions.py\", line 306, in wrapped_f\r\n    raise error(e.error_message, e.response_code, e.response_body) from e\r\ngitlab.exceptions.GitlabGetError: 404: 404 Project Not Found\r\n", "msg": "MODULE FAILURE\nSee stdout/stderr for the exact error", "rc": 1}

The same URL is working fine when using it with the Terraform provider or a simple cURL request like this:

curl --header "PRIVATE-TOKEN: <my_private_token>" "https://gitlab.local.fqdn/api/v4/projects/"
[
  {
    "variable_type": "env_var",
    "key": "ANSIBLE_USER",
    "value": "ansible",
    "protected": false,
    "masked": false,
    "environment_scope": "*"
  }
]

The behaviour is the same, whether run in the Gitlab-CI or on my local system.

Issue Type

Bug Report

Component Name

gitlab_project_variable

Ansible Version

$ ansible --version
ansible [core 2.11.1] 
  config file = /Users/hk/ansible-role-name/ansible.cfg
  configured module search path = ['/Users/hk/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /opt/homebrew/lib/python3.9/site-packages/ansible
  ansible collection location = /Users/hk/.ansible/collections:/usr/share/ansible/collections
  executable location = /opt/homebrew/bin/ansible
  python version = 3.9.5 (default, May  3 2021, 19:12:05) [Clang 12.0.5 (clang-1205.0.22.9)]
  jinja version = 3.0.1
  libyaml = False

Configuration

$ ansible-config dump --only-changed

OS / Environment

CentOS, macOS 11.4, Ubuntu 20.04

Steps to Reproduce

- name: write kubeconfig to Gitlab variables
      community.general.gitlab_project_variable:
        api_url: "https://gitlab.local.fqdn/api/v4"
        api_token: "{{ lookup('env', 'GITLAB_TOKEN') }}"
        project: "ansible/ansible-role-name"
        purge: false
        vars:
          KUBECONFIG: kubeconfig

Expected Results

The Gitlab project variable gets successfully created.

Actual Results

The full traceback is:
  File "/tmp/ansible_community.general.gitlab_project_variable_payload_ywq8urvk/ansible_community.general.gitlab_project_variable_payload.zip/ansible_collections/community/general/plugins/module_utils/gitlab.py", line 99, in gitlabAuthentication
    gitlab_instance.auth()
  File "/usr/local/lib/python3.8/dist-packages/gitlab/client.py", line 257, in auth
    self.user = self._objects.CurrentUserManager(self).get()
  File "/usr/local/lib/python3.8/dist-packages/gitlab/exceptions.py", line 306, in wrapped_f
    raise error(e.error_message, e.response_code, e.response_body) from e
fatal: [node_name_XY]: FAILED! => {
    "changed": false,
    "invocation": {
        "module_args": {
            "api_password": null,
            "api_token": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER",
            "api_url": "https://gitlab.local.fqdn/api/v4",
            "api_username": null,
            "project": "ansible/ansible-role-name",
            "purge": false,
            "state": "present",
            "validate_certs": true,
            "vars": {
                "KUBECONFIG": "VALUE_SPECIFIED_IN_NO_LOG_PARAMETER"
            }
        }
    },
    "msg": "Failed to connect to GitLab server: 404: 404 Not Found"
}

Code of Conduct

• [X] I agree to follow the Ansible Code of Conduct

[ansibullbot]

Files identified in the description:

• plugins/modules/source_control/gitlab/gitlab_project_variable.py

If these files are inaccurate, please update the component name section of the description or use the !component bot command.

click here for bot help

[ansibullbot]

cc @Lunik @Shaps @dj-wasabi @markuman @marwatk @metanovii @scodeman @waheedi @zanssa click here for bot help

[markuman]

"https://gitlab.local.fqdn" is the right value.

The error message is different, when calling "https://gitlab.local.fqdn", though:

The returning error message you've pasted 404: 404 Project Not Found indicates two possible errors.

1. the project does not exist (typo e.g.)
2. the provided credentials have insufficient permission

[kayahk]

Hi @markuman,

thank you for your suggestions. The project does exist, as I tried to make this clear by showing the cURL access result. The API token has the scope api and should therefore have all required privileges. Could you please clarify the exact requirement for the parameter project of this module? I find the documentation a bit unclear on this point.

project (string / required): The path and name of the project

However, when looking at the Gitlab API objects, I can only find these somewhat matching keys:

"id": 281
"name": "ansible-role-name",
"name_with_namespace": "TOP_GROUP/SUB_GROUP/Ansible/ansible-role-name",
"path": "ansible-role-name",
"path_with_namespace": "TOP_GROUP/SUB_GROUP/Ansible/ansible-role-name",
"namespace": {
      "id": 156,
      "name": "Ansible",
      "path": "ansible",
      "kind": "group",
      ...

Appreciate your comment on which of the API object keys would be the corresponding match for the Ansible parameter?

[kayahk]

@markuman can I provide anything else to help you help me with this issue? I would be happy to add more details if required.

[ansibullbot]

cc @sh0shin click here for bot help

[kinola]

Hello same error but to create a project My token user is ADMIN and i'm able to create project with bash script

"msg": "Failed to create project: group pac doesn't exists"

[ansibullbot]

cc @lgatellier @nejch click here for bot help

[markuman]

Can one of you reproduce this on gitlab.com?

[ansibullbot]

cc @suukit click here for bot help

[ansibullbot]

Files identified in the description:

• plugins/modules/gitlab_project_variable.py

If these files are incorrect, please update the component name section of the description or use the !component bot command.

click here for bot help

[lgatellier]

@kayahk @kinola can you reproduce the bug on gitlab.com ?

needs_info

[ansibullbot]

@kayahk This issue is waiting for your response. Please respond or the issue will be closed.

click here for bot help

[kayahk]

@lgatellier apologies for the delay, unfortunately I am currently out of any environment with access to Gitlab to reproduce the issue. Sorry, I am unable to help at the moment, but I will ask around my network if someone else can try to reproduce it.