Invalid credentials provided. Missing API key or an invalid API key was provided

[jrobison-sb]

Description

I'm attempting to use this role to install the New Relic infrastructure agent, and I have supplied my API key and my account ID, but I'm getting Invalid credentials provided. Missing API key or an invalid API key was provided when it gets to the Run CLI install task.

Steps to Reproduce

- name: Run the newrelic.newrelic_install role to install the newrelic agent
  # https://docs.newrelic.com/docs/infrastructure/install-infrastructure-agent/config-management-tools/configure-infrastructure-agent-using-ansible/
  include_role:
    name: newrelic.newrelic_install
    apply:
      environment:
        NEW_RELIC_API_KEY: "{{ tableau_newrelic_license_key }}"
        NEW_RELIC_ACCOUNT_ID: 123456
        NEW_RELIC_REGION: US
  vars:
    targets:
      - infrastructure
      - logs
    tags:
      environment: sandbox
      name: "sandbox-tableau"
      system: tableau

Expected Behavior

The Ansible role should install the agent without error.

Relevant Logs / Console output

fatal: [localhost]: FAILED! => {
    "ansible_job_id": "959606008271.92330",
    "changed": true,
    "cmd": "/usr/local/bin/newrelic install -y -n infrastructure-agent-installer,logs-integration --tag nr_deployed_by:ansible-install,environment:sandbox,name:sandbox-tableau,system:tableau",
    "delta": "0:04:58.576067",
    "end": "2023-08-18 19:16:05.419307",
    "finished": 1,
    "invocation": {
        "module_args": {
            "_raw_params": "/usr/local/bin/newrelic install -y -n infrastructure-agent-installer,logs-integration --tag nr_deployed_by:ansible-install,environment:sandbox,name:sandbox-tableau,system:tableau",
            "_uses_shell": true,
            "argv": null,
            "chdir": null,
            "creates": null,
            "executable": null,
            "removes": null,
            "stdin": null,
            "stdin_add_newline": true,
            "strip_empty_ends": true,
            "warn": false
        }
    },
    "msg": "non-zero return code",
    "rc": 1,
    "results_file": "/root/.ansible_async/959606008271.92330",
    "start": "2023-08-18 19:11:06.843240",
    "started": 1,
    "stderr": "level=fatal msg=\"could not fetch license key for account 123456:, license key:  401 response returned: Invalid credentials provided. Missing API key or an invalid API key was provided.\"",
    "stderr_lines": [
        "level=fatal msg=\"could not fetch license key for account 123456:, license key:  401 response returned: Invalid credentials provided. Missing API key or an invalid API key was provided.\""
    ],
    "stdout": "",
    "stdout_lines": []
}

Your Environment

• Ubuntu 20.04 on EC2 on AWS

ansible --version
ansible [core 2.13.11]
  config file = /etc/ansible/ansible.cfg
  configured module search path = ['/root/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
  ansible python module location = /usr/local/lib/python3.8/dist-packages/ansible
  ansible collection location = /root/.ansible/collections:/usr/share/ansible/collections
  executable location = /usr/local/bin/ansible
  python version = 3.8.10 (default, May 26 2023, 14:05:08) [GCC 9.4.0]
  jinja version = 3.1.2
  libyaml = True

Additional context

When I run this role in verbose mode using ansible-playbook -vvv, I can see in the verbose output that account ID and the API key are both being passed as environment variables and they appear to be the correct values. Eg:

ASYNC POLL on localhost: jid=959606008271.92330 started=1 finished=0
<127.0.0.1> EXEC /bin/sh -c 'echo ~root && sleep 0'
Using module file /usr/local/lib/python3.8/dist-packages/ansible/modules/async_status.py
<127.0.0.1> PUT /root/.ansible/tmp/ansible-local-76989117yt828/tmpigrobw28 TO /root/.ansible/tmp/ansible-tmp-1692385876.7790117-92311-55812131229027/AnsiballZ_async_status.py
<127.0.0.1> EXEC /bin/sh -c 'chmod u+x /root/.ansible/tmp/ansible-tmp-1692385876.7790117-92311-55812131229027/ /root/.ansible/tmp/ansible-tmp-1692385876.7790117-92311-55812131229027/AnsiballZ_async_status.py && sleep 0'
<127.0.0.1> EXEC /bin/sh -c 'NEW_RELIC_API_KEY=redactedapikeyishere NEW_RELIC_ACCOUNT_ID=123456 NEW_RELIC_REGION=US NEW_RELIC_CLI_SKIP_CORE=1 /usr/bin/python3 /root/.ansible/tmp/ansible-tmp-1692385876.7790117-92311-55812131229027/AnsiballZ_async_status.py && sleep 0'

I have obviously redacted my account number and API key in this issue. But both match what is displayed in https://one.newrelic.com/admin-portal/api-keys/home. The API key type is INGEST - LICENSE.

How can I troubleshoot this?

Thanks.

[jrobison-sb]

Nevermind, I'm unblocked.

Apparently the API key needs to be a User key, not a LICENSE - INGEST key. User keys apparently will have a NRAK prefix.

[burner1024]

But... why? Key documentation clearly states that user keys are not for that.

What madness is this?