solidify / jira-azuredevops-migrator

Tool to migrate work items from Atlassian Jira to Microsoft Azure DevOps/VSTS/TFS.
MIT License
262 stars 223 forks source link

Getting 401 Unauthorized with JIRA but no way to debug the root cause #730

Open hrvojeKol opened 1 year ago

hrvojeKol commented 1 year ago

I'm trying to use the tool to migrate project requirement data from corporate JIRA instance to corporate on-prem Azure DevOps server. I've setup all the neccessary configuration and the flow fails at the very start connecting to JIRA server with 401 error

Steps to reproduce the behavior:

  1. I setup the config file
  2. I get the Personal Access Token (PAT) created in JIRA profile
  3. I confirmed the PAT as valid, by sending a test request for a ticket using Postman

example request: GET https://{corporate JIRA server}/rest/api/latest/issue/{some issue ID} Response is 200 OK. Authorization is Oauth 2.0 with Bearer token as the PAT

  1. I start the script with .\jira-export.exe -u {my username} -p {PAT} --url https://{corporate JIRA server} --config {config file} --force

  2. I get
    Export complete. Exported 0 items (4 errors, 0 warnings) in 00:00:02. in the displayed payload, I get:

    Encountered a "401 - Unauthorized" error while loading this page.

Version of the jira-azuredevops-migrator tool is 3.0.115 JIRA verion v9.4.3

I've been testing few variations on the username parameter and even server/API URL,, but I just cannot seem to get it working. I'm aware there might be too little information and that this may be environment-specific, but is there any way to debug what is actually happening with the authentication/authorization, why can't this tool handle it?

Alexander-Hjelm commented 1 year ago

Tried authenticating with both your jira UserId, username and email?

hrvojeKol commented 1 year ago

Tried with username and email as stated in the ticket. Don't really know how to get the User ID from? (it's not visible in my Profile page in JIRA nor do I see it in any URL). Still, if I use the Personal Access Token, why is username even required? Shouldn't the token be enough for both authentication and authorization? Also, if it helps in any way, having using-jira-cloud set to true or false, makes no difference. Here it should be true I assume, since this is not an on-prem instance, although it's a corporate instance.

Alexander-Hjelm commented 1 year ago

Can you attach you jira export log?

hrvojeKol commented 1 year ago

I'm not sure I can attach the entire log due to compliance (I might be exposing some internal security information, but sure if there is any there..) But I can share snippets, i..e first few lines:

_[I][09:01:37] Connecting to Jira... [I][09:01:37] Retrieving Jira fields... [E][09:01:38] Failed to retrieve fields from Jira (Response was not recognized as JSON). This usually indicates a problem with authentication or authorization. Check your Jira credentials and permissions. [System.AggregateException] System.AggregateException: One or more errors occurred. ---> System.Security.Authentication.AuthenticationException: Response Content:

Unauthorized (401)_ ... and then some at the end: _at Atlassian.Jira.Remote.JiraRestClient.GetValidJsonFromResponse(IRestRequest request, IRestResponse response) at Atlassian.Jira.Remote.JiraRestClient.d__11.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Atlassian.Jira.Remote.JiraRestClient.d__10`1.MoveNext() --- End of stack trace from previous location where exception was thrown --- at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task) at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task) at Atlassian.Jira.Remote.IssueFieldService.d__2.MoveNext() --- End of inner exception stack trace --- at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions) at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken) at JiraExport.JiraProvider.Initialize(JiraSettings settings) ---> (Inner Exception #0) System.Security.Authentication.AuthenticationException: Response Content: _
hrvojeKol commented 1 year ago

I there any way to debug the communication and see why authentication fails? If not, I'm fine with closing close the issue but I would I propose that you add some more low-level debugging in the tool (i.e. network requests triggered, request/responses), aside from the standard logical export log.

Alexander-Hjelm commented 1 year ago

Yeah what you see in the log is the only response that Jira is sending. For jira cloud it should work out of the box.

using-jira-cloud should be false.

It could indicate that the url, user or token is invalid. Here is how I would invoke the exporter for example:

-u alexander.hjelm@solidify.dev -p xxxxxxxxxxxxxxxxxxxxxxxxxxxxx--url https://solidifydemo.atlassian.net/ --config C:\dev\jira\config-agile.json --force

If nothing works, check the older issues on this board, because this is not a too uncommon issue :)

Alexander-Hjelm commented 1 year ago

Or try a free TCP debugging tool like wireshark to inspect the data traffic and verify that URL and auth header looks correct.

tenderi commented 1 year ago

I am having identical issue. I can confirm that my PAT works if I just curl to my Jira server. The "Last authenticated" field updates in my personal access tokens -page with curl, but when I run the migrator, it does not - which seems to hint that the PAT is not even used.

hrvojeKol commented 1 year ago

I can confirm what @tenderi is sharing. Using Postman in my case to trigger some GET requests on a ticket, results in "Last authenticated" timestamp to update in my JIRA PAT. However, using the export tool doesn't change that value, something is definitely wrong with token usage. Not sure if network tracing would even help with this even since, the only parameter here is the URL of the instance, there's nothing more to configure as I can understand. As mentioned previously, a slightly more detailed debug log from the tool itself would be helpful in these situations, I think it's a bit too much of a black box for such non-trivial integration :)

Alexander-Hjelm commented 11 months ago

For anyone experiencing this issue in the future, I have notice a quirky behavior when running the tool from a powershell window. It seems that if the password/api token contains a dollar sign ($), the tool cannot properly parse the --password parameter. My solution was to escape any dollar sign characters like so:

`$

So make sure that your --password parameter is properly escaped, depending on what terminal you are using.

bb-nc283 commented 9 months ago

I got the exact same issue. Have tried everything suggested and all combinations, nothing works. When logging into Jira, I don't use e-mail in my username. When using no e-mail in username, I get a 403. When using e-mail I get a 401.

It should be stated that TOTP is enabled. I cannot disable this. But it should work with the PAT, as I'm able to use it as a bearer token when using Postman. As others have stated, there's no need to use a username, when using the PAT. Are you using the PAT as a bearer token?

403:

<div class="aui-page-panel" ><div class="aui-page-panel-inner">
        <main role="main" id="main" class="aui-page-panel-content" >
                <div class="aui-page-header" ><div class="aui-page-header-inner">
                        <div class="aui-page-header-main" >
                                <h1>Forbidden (403)</h1>
                            </div>
                    </div></div>
                <div class="aui-message aui-message-warning warning">
                        <p>Encountered a <code>&quot;403 - Forbidden&quot;</code> error while loading this page.</p>
                        <p>Basic Authentication Failure - Reason : AUTHENTICATION_DENIED</p>
                        <p><a href="/secure/MyJiraHome.jspa">Go to Jira home</a></p>
                    </div>
            </main>
    </div></div>