search

pycontribs / jira

Python Jira library. Development chat available on https://matrix.to/#/#pycontribs:matrix.org
https://jira.readthedocs.io
BSD 2-Clause "Simplified" License
1.96k stars 873 forks source link

basic_auth with API Key to Jira Cloud not working #1497

Open shaering-vdev opened 2 years ago

shaering-vdev commented 2 years ago

Bug summary

I've got multiple projects where I initialize Jira using basic_auth and an email / api token. This has been working fine until this morning, now I'm getting complaints about basic_auth being depreciated. I've tried using token_auth=TOKEN, but that doesn't work as I'm trying to hit a Cloud instance.

There is one project that already had version 3.2.0 installed locally, I am able to connect to Jira that with that version. However, when I specified 3.2.0 as the required version in a new environment, it was installed but connecting to the server still fails with the same error.

Is there a new method of connecting to Jira Cloud using an email / API key that I missed? Documentation still points me towards using basic_auth.

Thanks!!

Is there an existing issue for this?

Jira Instance type

Jira Cloud (Hosted by Atlassian)

Jira instance version

Current

jira-python version

main

Python Interpreter version

3.10

Which operating systems have you used?

Reproduction steps

self.jiraOptions = {'server': self.url, 'verify': 'auth/jira.pem'}
self.jira = JIRA(options=self.jiraOptions, basic_auth=(self.user, self.key))

Stack trace

File "C:\Users\shaering\OneDrive -  Corporation\Documents\GitHub\CLD-NaggerD\mod_jira.py", line 33, in __init__
    self.jira = JIRA(options=self.jiraOptions, basic_auth=(self.user, self.key))
  File "C:\Users\shaering\OneDrive -  Corporation\Documents\GitHub\CLD-NaggerD\venv\lib\site-packages\jira\client.py", line 571, in __init__
    si = self.server_info()
  File "C:\Users\shaering\OneDrive -  Corporation\Documents\GitHub\CLD-NaggerD\venv\lib\site-packages\jira\client.py", line 2962, in server_info
    j = self._get_json("serverInfo")
  File "C:\Users\shaering\OneDrive -  Corporation\Documents\GitHub\CLD-NaggerD\venv\lib\site-packages\jira\client.py", line 3624, in _get_json
    r = self._session.get(url, params=params)
  File "C:\Users\shaering\OneDrive -  Corporation\Documents\GitHub\CLD-NaggerD\venv\lib\site-packages\jira\resilientsession.py", line 195, in get
    return self.__verb("GET", str(url), **kwargs)
  File "C:\Users\shaering\OneDrive -  Corporation\Documents\GitHub\CLD-NaggerD\venv\lib\site-packages\jira\resilientsession.py", line 189, in __verb
    raise_on_error(response, verb=verb, **kwargs)
  File "C:\Users\shaering\OneDrive -  Corporation\Documents\GitHub\CLD-NaggerD\venv\lib\site-packages\jira\resilientsession.py", line 64, in raise_on_error
    raise JIRAError(
jira.exceptions.JIRAError: JiraError HTTP 401 url: https://infrastructure.atlassian.net/rest/api/2/serverInfo
    text: Basic authentication with passwords is deprecated.  For more information, see: https://developer.atlassian.com/cloud/confluence/deprecation-notice-basic-auth/

    response headers = {'Date': 'Wed, 07 Sep 2022 17:03:10 GMT', 'Content-Type': 'text/plain', 'Server': 'globaledge-envoy', 'X-Envoy-Upstream-Service-Time': '1', 'Expect-Ct': 'report-uri="https://web-security-reports.services.atlassian.com/expect-ct-report/atlassian-proxy", max-age=86400', 'Strict-Transport-Security': 'max-age=63072000; preload', 'X-Content-Type-Options': 'nosniff', 'X-Xss-Protection': '1; mode=block', 'Atl-Traceid': 'd5680b33bd315d2f', 'Report-To': '{"endpoints": [{"url": "https://dz8aopenkvv6s.cloudfront.net"}], "group": "endpoint-1", "include_subdomains": true, "max_age": 600}', 'Nel': '{"failure_fraction": 0.001, "include_subdomains": true, "max_age": 600, "report_to": "endpoint-1"}', 'Vary': 'Accept-Encoding', 'Content-Encoding': 'gzip', 'Transfer-Encoding': 'chunked'}
    response text = Basic authentication with passwords is deprecated.  For more information, see: https://developer.atlassian.com/cloud/confluence/deprecation-notice-basic-auth/

Expected behaviour

Authenticate to Jira without issues using an email / api key

Additional Context

beautifulsoup4==4.11.1 bs4==0.0.1 certifi==2022.6.15 charset-normalizer==2.1.0 defusedxml==0.7.1 html5lib==1.1 idna==3.3 oauthlib==3.2.0 packaging==21.3 ping3==4.0.3 pyparsing==3.0.9 python-dotenv==0.20.0 requests==2.28.1 requests-oauthlib==1.3.1 requests-toolbelt==0.9.1 six==1.16.0 soupsieve==2.3.2.post1 typing_extensions==4.3.0 urllib3==1.26.10 webencodings==0.5.1

adehad commented 2 years ago

Sorry, can I confirm you have tried the latest release of 3.4.1 ?

shaering-vdev commented 2 years ago

Correct, I tried with it just now, and received the same error.

beautifulsoup4==4.11.1 bs4==0.0.1 certifi==2022.6.15 charset-normalizer==2.1.0 defusedxml==0.7.1 html5lib==1.1 idna==3.3 jira==3.4.1 oauthlib==3.2.0 packaging==21.3 ping3==4.0.3 pyparsing==3.0.9 python-dotenv==0.20.0 requests==2.28.1 requests-oauthlib==1.3.1 requests-toolbelt==0.9.1 six==1.16.0 soupsieve==2.3.2.post1 typing_extensions==4.3.0 urllib3==1.26.10 webencodings==0.5.1

shaering-vdev commented 2 years ago

Hey guys, any feedback? I'm still running into the same issue, just in environments with the latest updates. The older environments I have set up that aren't updated are fine.

adehad commented 2 years ago

Unfortunately haven't had the chance to look into reproducing this.

yxuko commented 2 years ago

Hey guys, I am also experiencing same issue with Jira Cloud. The method basic_auth no longer works with token.

princenyeche commented 2 years ago

@shaering-vdev The easiest way to log in to a cloud instance. That works for me, on v3.4.1 and even the previous version you mentioned

from jira import JIRA
my_jira = JIRA(server="https://example.atlassian.net",
                 basic_auth=("prince@example.com",
                 "MXKSlsXXXXX"))

I'm not sure why you had to specify 'verify': 'auth/jira.pem' in your reproduction steps to a cloud instance

self.jiraOptions = {'server': self.url, 'verify': 'auth/jira.pem'}
joristork commented 2 years ago

Two hours ago my JIRA(server=<server>, basic_auth=(<email>, <api token>)) worked fine. Now I'm getting Basic authentication with passwords is deprecated.. I've tried generating a fresh API token and using that, same result. I've tried switching to token_auth=<api token> and get Session auth token was empty errors. I'm running v3.4.1.

adehad commented 1 year ago

Two hours ago my JIRA(server=<server>, basic_auth=(<email>, <api token>)) worked fine. Now I'm getting Basic authentication with passwords is deprecated.. I've tried generating a fresh API token and using that, same result. I've tried switching to token_auth=<api token> and get Session auth token was empty errors. I'm running v3.4.1.

@joristork - just to confirm my understanding, on the same version of the package (ie 3.4.1) you lost the ability to login with basic auth?

adehad commented 1 year ago

I think it would be great if we can confirm whether this snippet works (stripping out the jira library, adapted from jira docs)

At the very least this will confirm it is a problem with the jira lib and not any changes in the requests library or the jira instance itself.

import requests
from requests.auth import HTTPBasicAuth
import json

url = "https://your-domain.atlassian.net/rest/api/2/serverInfo"
auth = ("email@example.com", "<api_token>")

headers = {
   "Accept": "application/json"
}
response = requests.request(
   "GET",
   url,
   headers=headers,
   auth=HTTPBasicAuth(*auth)
)

session = requests.Session()
session.auth = auth

response_session = session.request(
   "GET",
   url,
   headers=headers,
)

assert response.text == response_session.text, f"{response.txt} \n\nvs\n\n {response_session.text}"

print(json.dumps(json.loads(response.text), sort_keys=True, indent=4, separators=(",", ": ")))
matheusleal-eb commented 1 year ago

I got this error message after running this snippet

    assert response == response_session
AssertionError

I just started facing this issue now but maybe we need to bump the default auth_url value to version 2

    DEFAULT_OPTIONS = {
        "server": "http://localhost:2990/jira",
        "auth_url": "/rest/auth/1/session",
        "context_path": "/",
        "rest_path": "api",
        "rest_api_version": "2",
        "agile_rest_path": AgileResource.AGILE_BASE_REST_PATH,
        "agile_rest_api_version": "1.0",
        "verify": True,
        "resilient": True,
        "async": False,
        "async_workers": 5,
        "client_cert": None,
        "check_update": False,
        # amount of seconds to wait for loading a resource after updating it
        # used to avoid server side caching issues, used to be 4 seconds.
        "delay_reload": 0,
        "headers": {
            "Cache-Control": "no-cache",
            # 'Accept': 'application/json;charset=UTF-8',  # default for REST
            "Content-Type": "application/json",  # ;charset=UTF-8',
            # 'Accept': 'application/json',  # default for REST
            # 'Pragma': 'no-cache',
            # 'Expires': 'Thu, 01 Jan 1970 00:00:00 GMT'
            "X-Atlassian-Token": "no-check",
        },
        "default_batch_size": {
            Resource: 100,
        },
    }
adehad commented 1 year ago

@matheusleal-eb Oops made a mistake in my snippet. I've updated it, but also added the correct assertion below

assert response.text == response_session.text, f"{response.txt} \n\nvs\n\n {response_session.text}"

It is true that the v1 auth_url is deprecated, but that code path is actually not run by default. Do you have your stack trace? My guess is that it is stuck on the serverInfo endpoint

matheusleal-eb commented 1 year ago

It works now 😄 

{
    "baseUrl": "https://XXXXXXX.atlassian.net",
    "buildDate": "2022-11-14T14:27:18.000-0800",
    "buildNumber": 100210,
    "defaultLocale": {
        "locale": "en_US"
    },
    "deploymentType": "Cloud",
    "scmInfo": "XXXXXXXXXXXX",
    "serverTitle": "Jira",
    "version": "1001.0.0-SNAPSHOT",
    "versionNumbers": [
        1001,
        0,
        0
    ]
}

Is there a quick way to bump the auth_url value to version 2 ?

[INFO]  2022-11-14T17:54:52.834Z    Trying to refresh the cookie auth session...
[ERROR] 2022-11-14T17:54:53.269Z    Error creating jira ticket
[ERROR] 2022-11-14T17:54:53.270Z    JiraError HTTP 401 url: https://XXXXXXX.atlassian.net/rest/auth/1/session
    text: Login failed

    response headers = {'Date': 'Mon, 14 Nov 2022 17:54:53 GMT', 'Content-Type': 'application/json;charset=UTF-8', 'Server': 'AtlassianEdge', 'Timing-Allow-Origin': '*', 'X-Arequestid': 'XXXXXX', 'Set-Cookie': 'atlassian.xsrf.token=XXXXXXXXXXX; path=/; SameSite=None; Secure', 'Www-Authenticate': 'JIRA REST POST, OAuth realm="https%3A%2F%2FXXXXXXX.atlassian.net"', 'Cache-Control': 'no-cache, no-store, no-transform', 'Expect-Ct': 'report-uri="https://web-security-reports.services.atlassian.com/expect-ct-report/atlassian-proxy", max-age=86400', 'Strict-Transport-Security': 'max-age=63072000; preload', 'X-Content-Type-Options': 'nosniff', 'X-Xss-Protection': '1; mode=block', 'Atl-Traceid': 'XXXXXX', 'Report-To': '{"endpoints": [{"url": "https://XXXXXX.cloudfront.net"}], "group": "endpoint-1", "include_subdomains": true, "max_age": 600}', 'Nel': '{"failure_fraction": 0.001, "include_subdomains": true, "max_age": 600, "report_to": "endpoint-1"}', 'Transfer-Encoding': 'chunked'}
    response text = {"errorMessages":["Login failed"],"errors":{}}
adehad commented 1 year ago

@matheusleal-eb From your log message it looks like you are using cookie auth? This definitely won't work with Jira Cloud as that has been deprecated.

Can you confirm your experience if using JIRA(server=<server>, basic_auth=(<email>, <api token>)) does not work (when using the API key)?

If so that points to the jira lib not correctly handling the auth in the session. (which is strange !)

Is there a quick way to bump the auth_url value to version 2 ?

Technically yes.

BUT my understanding is this endpoint is removed (per my earlier link).

If you do want to try it, you would be looking to override: https://jira.readthedocs.io/api.html#jira.client.JIRA.DEFAULT_OPTIONS

specifically the 'auth_url' key. The override should be passed in as a dictionary to the options argument to the JIRA client.

adehad commented 1 year ago

From the original report and it sounds like there is a dependency somewhere that has been updated that is causing an issue. As we lock our dependencies we may not be seeing it in our pipelines?

matheusleal-eb commented 1 year ago

Using JIRA(server=<server>, basic_auth=(<email>, <api token>)) doesn't work either (when using the API key), but I get another error:

text: Basic authentication with passwords is deprecated.  For more information, see: https://developer.atlassian.com/cloud/confluence/deprecation-notice-basic-auth/
adehad commented 1 year ago

@matheusleal-eb thanks for confirming, I have some capacity to look into this, so I'll see if there is anything I can find.

From the original report and it sounds like there is a dependency somewhere that has been updated that is causing an issue. As we lock our dependencies we may not be seeing it in our pipelines?

Our cloud tests are working and I have bumped the dependencies using dependabot, so I'm not convinced this is the case. Likely a bad refactor on my part

princenyeche commented 1 year ago

Hey, @matheusleal-eb are you sure that you've generated an API token from this link for cloud instance and not the admin API key as those are a different token entirely? I see no problem with the current version or the previous one, so if it has stopped working for you, then there's something wrong with what you're sending in the request that's causing the authentication to fail.

adehad commented 1 year ago

The update I can provide is that when using an incorrect API key, it returns the same error as using the password. Which I understand, but does also make it a bit harder to debug the issue.

eg using our instance

j = JIRA(
    server="https://pycontribs.atlassian.net",
    basic_auth=(
        "ci-user@spadehadm.33mail.com",
        "incorrect",
    ),
)

Interesting point there @princenyeche, I would be curious if that is the issue, as when using the snippet from earlier https://github.com/pycontribs/jira/issues/1497#issuecomment-1313632749, (I assume with the same key), @matheusleal-eb can get a correct response from the server.

princenyeche commented 1 year ago

@adehad that endpoint https://your-domain.atlassian.net/rest/api/2/serverInfo is accessible anonymously, so a 200 response will be returned even though the authentication is wrong. They need to thoroughly test their API token and check for the response to know what status_code is returned. Another thing is that if they are certain the API token is correct, it should return a status code other than 401; if it still returns a 401 status code then it's definitely wrong. If it's the former, they can then check that they've not blocked any Atlassian IPs and domain ranges.

eNcacz commented 1 year ago

For future readers which come here from a search engine:

I had the same issue today. I have two projects which use basic_auth with username and token. One project worked properly, the other one raised the "Basic authentication with passwords is deprecated" error.

The token (as well as username) in the wrong project was ... wrong. They contained proper value but including double quotes around the value. It is stupid mistake but it is hard to see it.

I hope it can help to somebody else and save some time of his life :-)