search

turbot / steampipe-plugin-jira

Use SQL to instantly query Jira. Open source CLI. No DB required.
https://hub.steampipe.io/plugins/turbot/jira
Apache License 2.0
22 stars 14 forks source link

Querying issues loads all issues #46

Closed pdecat closed 2 years ago

pdecat commented 2 years ago

Describe the bug

When querying issues, no filter is passed to the API and all issues are requested from the JIRA API.

With a large number of issues, this can take a very long time, and maybe never succeed.

I had to add traces to the plugin to understand what was going on:

2022-02-23 16:09:43.334 UTC [DEBUG] steampipe-plugin-jira.plugin: [DEBUG] Steampipe API Request Details:
2022-02-23 16:09:43.334 UTC [DEBUG] steampipe-plugin-jira.plugin: ---[ REQUEST ]---------------------------------------
2022-02-23 16:09:43.334 UTC [DEBUG] steampipe-plugin-jira.plugin: GET /rest/api/2/search?startAt=0&maxResults=1000&expand=names HTTP/1.1
2022-02-23 16:09:43.334 UTC [DEBUG] steampipe-plugin-jira.plugin: Host: mydomain.atlassian.net
2022-02-23 16:09:43.334 UTC [DEBUG] steampipe-plugin-jira.plugin: User-Agent: Go-http-client/1.1
2022-02-23 16:09:43.334 UTC [DEBUG] steampipe-plugin-jira.plugin: Content-Type: application/json
2022-02-23 16:09:43.334 UTC [DEBUG] steampipe-plugin-jira.plugin: Accept-Encoding: gzip
2022-02-23 16:09:43.334 UTC [DEBUG] steampipe-plugin-jira.plugin:
2022-02-23 16:09:43.334 UTC [DEBUG] steampipe-plugin-jira.plugin:
2022-02-23 16:09:43.334 UTC [DEBUG] steampipe-plugin-jira.plugin: -----------------------------------------------------
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: [DEBUG] Steampipe API Response Details:
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: ---[ RESPONSE ]--------------------------------------
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: HTTP/2.0 200 OK
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: Atl-Traceid: ******
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: Cache-Control: no-cache, no-store, no-transform
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: Content-Type: application/json;charset=UTF-8
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: Date: Wed, 23 Feb 2022 16:09:46 GMT
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: Expect-Ct: report-uri="https://web-security-reports.services.atlassian.com/expect-ct-report/global-proxy", enforce, max-age=86400
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: Server: AtlassianProxy/1.19.3.1
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: Set-Cookie: atlassian.xsrf.token=******; path=/; SameSite=None; Secure
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: Strict-Transport-Security: max-age=315360000; includeSubDomains; preload
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: Timing-Allow-Origin: *
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: Vary: Accept-Encoding
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: X-Aaccountid: ******
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: X-Arequestid: ******
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: X-Content-Type-Options: nosniff
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: X-Envoy-Upstream-Service-Time: 2753
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: X-Xss-Protection: 1; mode=block
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin:
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin: {
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin:  "expand": "schema,names",
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin:  "startAt": 0,
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin:  "maxResults": 100,
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin:  "total": 505555,
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin:  "issues": [
2022-02-23 16:09:47.480 UTC [DEBUG] steampipe-plugin-jira.plugin:   {
...

Steampipe version (steampipe -v)

v0.12.2

Plugin version (steampipe plugin list)

v0.1.0

To reproduce

steampipe query "select id, key from jira_issue where key = 'MYPROJECT-42';"

Relevant code is https://github.com/turbot/steampipe-plugin-jira/blob/v0.1.0/jira/table_jira_issue.go#L221-L225

Expected behavior

A filter should be added to the JIRA API request to avoid retrieving all issues.

Additional context N/A

LalitLab commented 2 years ago

@pdecat Thanks for reporting this issue! Sorry, it's taken so long for us to respond, we missed this issue when it was first opened.

We've identified below additional key columns: issue_id, issue_key, project_key, project_id, project_name, status, assignee_display_name, assignee_display_name, creator_account_id, creator_display_name, reporter_account_id, reporter_display_name, created, duedate, updated, epic_key, type and priority.

We'll start work on adding these this week, and they should be ready later early next week in a new plugin version.

cbruno10 commented 2 years ago

Hey @pdecat , optional key qualifiers have been added to the jira_issue table in v0.2.0 (full list available here), which should allow for more efficient querying.

If you're still seeing slow queries or other issues, please let us know!