get-JiraIssue is more than 10-20 times slower than the API

sgtwilko commented 5 years ago

Description

I'm using get-JiraIssue with a query. When I run the query in Jira, or if I directly call the Jira API the results are returned in 1 to 4 seconds. When running the same query in JiraPS the results take 30-50 seconds to return. If I limit the fields being returned in JiraPS the results take 10-15 seconds.

When calling the API I did not select specific fields, I left it at the default.

Steps To Reproduce

We are using the standard get-JiraIssue and passing in a query: $jiras = get-jiraissue -query $jql -cred $cred

The results are returned very very slowly.

If we don't limit the fields we see JiraPS being 25 times slower than the API. By limiting the results it is only 10-15 times slower than the API.

issueFunction in parentsOf('project =\"My Project Name\" and sprint=1000 and type in subTaskIssueTypes() and type not in(\"Testing Task\",\"Sub Test Execution\") and assignee=user.name') and status != Closed and issueFunction in completeInSprint("MPN","MPN Sprint 1")

Jira API: 3.4 seconds get-JiraIssue: 94 seconds get-JiraIssue - fields: 31 seconds

Expected behavior

The results to be returned in almost the same time as running the query directly in the API, with only a small overhead.

Your Environment

Name   Version
----   -------
JiraPS 2.11.1

Name                           Value                                                                                                                                                                        
----                           -----                                                                                                                                                                        
PSVersion                      5.1.17134.765                                                                                                                                                                
PSEdition                      Desktop                                                                                                                                                                      
PSCompatibleVersions           {1.0, 2.0, 3.0, 4.0...}                                                                                                                                                      
BuildVersion                   10.0.17134.765                                                                                                                                                               
CLRVersion                     4.0.30319.42000                                                                                                                                                              
WSManStackVersion              3.0                                                                                                                                                                          
PSRemotingProtocolVersion      2.3                                                                                                                                                                          
SerializationVersion           1.1.0.1

Possible Solution

I think it's fairly obvious that the number of fields we have is slowing things down, but somehow the python jira library loads and processes the results far faster, there must be some processing of the results into objects that is taking far longer than necessary.

it would seem to be happening between the two debug statements below:

DEBUG: [Test-ServerResponse] Investigating $InputObject.Headers['X-Seraph-LoginReason']
...
DEBUG: Extracting data from [issues] containter

lipkau commented 5 years ago

Thank you for reporting this. The only command I see that consumes a lot of resources between those to debug messages is this: https://github.com/AtlassianPS/JiraPS/blob/9786aa03798de8a75b4ebdb897b5729472a88f13/JiraPS/Public/Invoke-JiraMethod.ps1#L194

could you test how long this conversion is taking for your data set?

sgtwilko commented 4 years ago

Hi @lipkau

I finally managed to do some testing.

Have you issued an update as this seems to be slightly quicker than my tests from a couple of weeks back? However this is still taking a very long time.

I've measured the API calls and am still seeing an average time for Jira to respond to the API call of ~3.5 seconds.

Below is an indicative run.

Total time spent was ~24 seconds:

DEBUG: [Invoke-JiraMethod] Starting Invoke-WebRequest
DEBUG: [Invoke-JiraMethod] Done Invoke-WebRequest in 4.1108596
DEBUG: [Invoke-JiraMethod] Executed WebRequest. Access $webResponse to see details
DEBUG: [Test-ServerResponse] Investigating $InputObject.Headers['X-Seraph-LoginReason']
DEBUG: [Invoke-JiraMethod] Starting Json
DEBUG: [Invoke-JiraMethod] Done Json in 12.0449712

In summary: less than 1 second before the Invoke-WebRequest was called, 4 seconds in Invoke-WebRequest (I heard this can also be slow), 12 seconds in ConvertFrom-Json, the remaining 7 to 8 seconds converting the objects in the ConvertTo-Jira*

AtlassianPS / JiraPS