tenable / integration-jira-cloud

69 stars 51 forks source link

Cannot setup new project, fails for "No project could be found with key 'VULN'." #90

Closed patrickoneiltc closed 3 years ago

patrickoneiltc commented 3 years ago

We had to rebuild our jira project. When I run the tenable-jira script, It errors out that the project doesn't exist, for a new setup. Actually, it just gives a generic keyerror, and inserting a print before the exception throwing code says that "No project could be found with key 'VULN'." I thought I could use the --setup-only version, but it creates a yaml file, which is not supported by Jira Cloud.

Here's the troubleshooting output.

LOOK HERE: {'errorMessages': ["No project could be found with key 'VULN'."], 'errors': {}}

Configuration File:

jira:
  address: <REDACTED>
  api_token: <REDACTED>
  api_username: <REDACTED>
log:
  filename: /var/log/tenable.log
  format: '%(asctime)-15s %(name)s %(levelname)s %(message)s'
  level: 20
project:
  key: VULN
  leadAccountId: <REDACTED>
  name: Vulnerability Management
service:
  interval: 1
tenable:
  access_key: <REDACTED>
  platform: tenable.io
  secret_key: <REDACTED>
  tio_age: 3
  tio_severities:
  - high
  - critical

Debug Logs

2021-01-13 17:46:40,096 root INFO Tenable2JiraCloud Version 1.1.18
2021-01-13 17:46:40,096 root INFO Using configuration file config.yaml
2021-01-13 17:46:40,096 root INFO Running on Python 3.6.9 Linux/x86_64
2021-01-13 17:46:40,097 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/session", "params": {}, "body": {}}
2021-01-13 17:46:40,099 urllib3.connectionpool DEBUG Starting new HTTPS connection (1): cloud.tenable.com:443
2021-01-13 17:46:40,399 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /session HTTP/1.1" 200 None
2021-01-13 17:46:40,400 tenable.io.TenableIO DEBUG Request-UUID 024264a9d153ab427d632493e0f96fe6 for https://cloud.tenable.com/session
2021-01-13 17:46:40,401 urllib3.connectionpool DEBUG Starting new HTTPS connection (1): <JIRA_CLOUD_HOST>:443
2021-01-13 17:46:40,481 urllib3.connectionpool DEBUG https://<JIRA_CLOUD_HOST>:443 "GET /rest/api/3/project/VULN HTTP/1.1" 404 None
2021-01-13 17:46:40,484 urllib3.connectionpool DEBUG Starting new HTTPS connection (1): <JIRA_CLOUD_HOST>:443
2021-01-13 17:46:40,655 urllib3.connectionpool DEBUG https://<JIRA_CLOUD_HOST>:443 "GET /rest/api/3/field HTTP/1.1" 200 None
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO Tenable Platform is customfield_11593 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO CVEs is customfield_11594 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO CVSSv2 Base Score is customfield_11595 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO CVSSv2 Temporal Score is customfield_11596 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO CVSSv3 Base Score is customfield_11597 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO CVSSv3 Temporal Score is customfield_11598 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO Patch Publication Date is customfield_11599 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO Tenable Plugin ID is customfield_11600 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO Tenable Plugin Family is customfield_11601 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO Tenable Plugin Name is customfield_11602 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO Vulnerability Severity is customfield_11603 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO Tenable Asset UUID is customfield_11604 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO Tenable Asset Tags is customfield_11605 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO Device MAC Addresses is customfield_11606 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO Device IPv4 Addresses is customfield_11607 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO Device IPv6 Addresses is customfield_11608 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO Device Hostname is customfield_11609 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO Device NetBIOS Name is customfield_11610 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO Device DNS Name is customfield_11611 (existing)
2021-01-13 17:46:40,661 tenable_jira.jira.Jira INFO Device Network ID is customfield_11612 (existing)
2021-01-13 17:46:40,662 tenable_jira.jira.Jira INFO Vulnerability First Seen is customfield_11613 (existing)
2021-01-13 17:46:40,662 tenable_jira.jira.Jira INFO Vulnerability Last Seen is customfield_11614 (existing)
2021-01-13 17:46:40,662 tenable_jira.jira.Jira INFO Vulnerability Last Fixed is customfield_11615 (existing)
2021-01-13 17:46:40,662 tenable_jira.jira.Jira INFO Vulnerability State is customfield_11616 (existing)
2021-01-13 17:46:40,662 tenable_jira.jira.Jira INFO Vulnerability Port is customfield_11617 (existing)
2021-01-13 17:46:40,662 tenable_jira.jira.Jira INFO Vulnerability Protocol is customfield_11618 (existing)
2021-01-13 17:46:40,662 tenable_jira.jira.Jira INFO Vulnerability Repository ID is customfield_11619 (existing)
2021-01-13 17:46:40,662 tenable_jira.jira.Jira INFO Vulnerability Repository Name is customfield_11620 (existing)
2021-01-13 17:46:40,662 tenable_jira.jira.Jira INFO Tenable VPR Score is customfield_11621 (existing)
2021-01-13 17:46:40,663 urllib3.connectionpool DEBUG Starting new HTTPS connection (1): <JIRA_CLOUD_HOST>:443
2021-01-13 17:46:40,723 urllib3.connectionpool DEBUG https://<JIRA_CLOUD_HOST>:443 "GET /rest/api/3/issuetype HTTP/1.1" 200 None
2021-01-13 17:46:40,727 urllib3.connectionpool DEBUG Starting new HTTPS connection (1): <JIRA_CLOUD_HOST>:443
2021-01-13 17:46:40,804 urllib3.connectionpool DEBUG https://<JIRA_CLOUD_HOST>:443 "GET /rest/api/3/screens?startAt=-100&maxResults=100 HTTP/1.1" 200 None
2021-01-13 17:46:40,808 urllib3.connectionpool DEBUG Starting new HTTPS connection (1): <JIRA_CLOUD_HOST>:443
2021-01-13 17:46:40,867 urllib3.connectionpool DEBUG https://<JIRA_CLOUD_HOST>:443 "GET /rest/api/3/screens?startAt=0&maxResults=100 HTTP/1.1" 200 None
2021-01-13 17:46:40,870 tenable_jira.transform.Tio2Jira INFO Using JIRA Screens []
2021-01-13 17:46:40,871 tenable.io.TenableIO DEBUG {"method": "POST", "url": "https://cloud.tenable.com/assets/export", "params": {}, "body": {"filters": {"updated_at": 1610236800}, "chunk_size": 1000}}
2021-01-13 17:46:41,799 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "POST /assets/export HTTP/1.1" 200 None
2021-01-13 17:46:41,800 tenable.io.TenableIO DEBUG Request-UUID fe7ade91f8d1eccc5b530aa063b04b14 for https://cloud.tenable.com/assets/export
2021-01-13 17:46:41,800 tenable.io.TenableIO DEBUG Initiated asset export b7c406f7-68ac-4aa5-93f6-52074f43aef9
2021-01-13 17:46:41,800 tenable.io.TenableIO DEBUG {"method": "POST", "url": "https://cloud.tenable.com/assets/export", "params": {}, "body": {"filters": {"deleted_at": 1610236800}, "chunk_size": 1000}}
2021-01-13 17:46:42,496 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "POST /assets/export HTTP/1.1" 200 None
2021-01-13 17:46:42,496 tenable.io.TenableIO DEBUG Request-UUID d9212805650f5fc03f8313701e3799a1 for https://cloud.tenable.com/assets/export
2021-01-13 17:46:42,496 tenable.io.TenableIO DEBUG Initiated asset export 7cfc7cb3-43ea-4585-b207-6a46f4e926b6
2021-01-13 17:46:42,497 tenable.io.TenableIO DEBUG {"method": "POST", "url": "https://cloud.tenable.com/assets/export", "params": {}, "body": {"filters": {"terminated_at": 1610236800}, "chunk_size": 1000}}
2021-01-13 17:46:44,111 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "POST /assets/export HTTP/1.1" 200 None
2021-01-13 17:46:44,112 tenable.io.TenableIO DEBUG Request-UUID 555f5d6d03f9c47bc7eac4363fe0b107 for https://cloud.tenable.com/assets/export
2021-01-13 17:46:44,112 tenable.io.TenableIO DEBUG Initiated asset export 4bf74b30-cd36-4852-bb58-1a3d368b25bf
2021-01-13 17:46:44,112 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/4bf74b30-cd36-4852-bb58-1a3d368b25bf/status", "params": {}, "body": {}}
2021-01-13 17:46:44,660 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/4bf74b30-cd36-4852-bb58-1a3d368b25bf/status HTTP/1.1" 200 None
2021-01-13 17:46:44,661 tenable.io.TenableIO DEBUG Request-UUID f0e2796a6721d5ee7ceab3920e99a797 for https://cloud.tenable.com/assets/export/4bf74b30-cd36-4852-bb58-1a3d368b25bf/status
2021-01-13 17:46:44,661 tenable.io.exports.ExportsIterator DEBUG EXPORT assets 4bf74b30-cd36-4852-bb58-1a3d368b25bf is status PROCESSING
2021-01-13 17:46:46,663 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/4bf74b30-cd36-4852-bb58-1a3d368b25bf/status", "params": {}, "body": {}}
2021-01-13 17:46:46,813 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/4bf74b30-cd36-4852-bb58-1a3d368b25bf/status HTTP/1.1" 200 None
2021-01-13 17:46:46,814 tenable.io.TenableIO DEBUG Request-UUID 0381370ad3f9e427c01e67a14af77a3f for https://cloud.tenable.com/assets/export/4bf74b30-cd36-4852-bb58-1a3d368b25bf/status
2021-01-13 17:46:46,814 tenable.io.exports.ExportsIterator DEBUG EXPORT assets 4bf74b30-cd36-4852-bb58-1a3d368b25bf is status FINISHED
2021-01-13 17:46:46,814 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/4bf74b30-cd36-4852-bb58-1a3d368b25bf/chunks/1", "params": {}, "body": {}}
2021-01-13 17:46:48,166 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/4bf74b30-cd36-4852-bb58-1a3d368b25bf/chunks/1 HTTP/1.1" 200 None
2021-01-13 17:46:48,167 tenable.io.TenableIO DEBUG Request-UUID 13d558899ec0574f477695d3c6890157 for https://cloud.tenable.com/assets/export/4bf74b30-cd36-4852-bb58-1a3d368b25bf/chunks/1
2021-01-13 17:46:48,169 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/4bf74b30-cd36-4852-bb58-1a3d368b25bf/status", "params": {}, "body": {}}
2021-01-13 17:46:49,214 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/4bf74b30-cd36-4852-bb58-1a3d368b25bf/status HTTP/1.1" 200 None
2021-01-13 17:46:49,215 tenable.io.TenableIO DEBUG Request-UUID 8d7eaca4b859f97ecb7a30c879b5abdd for https://cloud.tenable.com/assets/export/4bf74b30-cd36-4852-bb58-1a3d368b25bf/status
2021-01-13 17:46:49,215 tenable.io.exports.ExportsIterator DEBUG EXPORT assets 4bf74b30-cd36-4852-bb58-1a3d368b25bf is status FINISHED
2021-01-13 17:46:49,216 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/7cfc7cb3-43ea-4585-b207-6a46f4e926b6/status", "params": {}, "body": {}}
2021-01-13 17:46:50,421 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/7cfc7cb3-43ea-4585-b207-6a46f4e926b6/status HTTP/1.1" 200 None
2021-01-13 17:46:50,422 tenable.io.TenableIO DEBUG Request-UUID 9811d76e22b8721f43354ee9397ba6dc for https://cloud.tenable.com/assets/export/7cfc7cb3-43ea-4585-b207-6a46f4e926b6/status
2021-01-13 17:46:50,422 tenable.io.exports.ExportsIterator DEBUG EXPORT assets 7cfc7cb3-43ea-4585-b207-6a46f4e926b6 is status FINISHED
2021-01-13 17:46:50,422 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status", "params": {}, "body": {}}
2021-01-13 17:46:51,684 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status HTTP/1.1" 200 None
2021-01-13 17:46:51,684 tenable.io.TenableIO DEBUG Request-UUID 09eaad4a3d3489e17ee01148d41091b2 for https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status
2021-01-13 17:46:51,684 tenable.io.exports.ExportsIterator DEBUG EXPORT assets b7c406f7-68ac-4aa5-93f6-52074f43aef9 is status QUEUED
2021-01-13 17:46:53,687 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status", "params": {}, "body": {}}
2021-01-13 17:46:53,846 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status HTTP/1.1" 200 None
2021-01-13 17:46:53,847 tenable.io.TenableIO DEBUG Request-UUID 3c3ef8777b67034d510b8de3fee9f48f for https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status
2021-01-13 17:46:53,847 tenable.io.exports.ExportsIterator DEBUG EXPORT assets b7c406f7-68ac-4aa5-93f6-52074f43aef9 is status QUEUED
2021-01-13 17:46:56,849 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status", "params": {}, "body": {}}
2021-01-13 17:46:57,039 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status HTTP/1.1" 200 None
2021-01-13 17:46:57,040 tenable.io.TenableIO DEBUG Request-UUID 281f732187eaf239285d7ceff14f3b2b for https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status
2021-01-13 17:46:57,040 tenable.io.exports.ExportsIterator DEBUG EXPORT assets b7c406f7-68ac-4aa5-93f6-52074f43aef9 is status QUEUED
2021-01-13 17:47:01,044 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status", "params": {}, "body": {}}
2021-01-13 17:47:01,263 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status HTTP/1.1" 200 None
2021-01-13 17:47:01,264 tenable.io.TenableIO DEBUG Request-UUID bc6acbda86131cb8cdaaec6f65bbeef1 for https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status
2021-01-13 17:47:01,264 tenable.io.exports.ExportsIterator DEBUG EXPORT assets b7c406f7-68ac-4aa5-93f6-52074f43aef9 is status QUEUED
2021-01-13 17:47:06,269 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status", "params": {}, "body": {}}
2021-01-13 17:47:06,614 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status HTTP/1.1" 200 None
2021-01-13 17:47:06,615 tenable.io.TenableIO DEBUG Request-UUID 9b291647fb1a94e1c7c5c263fbbbb2c6 for https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status
2021-01-13 17:47:06,615 tenable.io.exports.ExportsIterator DEBUG EXPORT assets b7c406f7-68ac-4aa5-93f6-52074f43aef9 is status QUEUED
2021-01-13 17:47:12,619 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status", "params": {}, "body": {}}
2021-01-13 17:47:12,987 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status HTTP/1.1" 200 None
2021-01-13 17:47:12,988 tenable.io.TenableIO DEBUG Request-UUID b288a126c6155c5e6fbb0d1f0818f727 for https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status
2021-01-13 17:47:12,988 tenable.io.exports.ExportsIterator DEBUG EXPORT assets b7c406f7-68ac-4aa5-93f6-52074f43aef9 is status QUEUED
2021-01-13 17:47:19,995 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status", "params": {}, "body": {}}
2021-01-13 17:47:20,204 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status HTTP/1.1" 200 None
2021-01-13 17:47:20,205 tenable.io.TenableIO DEBUG Request-UUID 4f2c3f18f59a9eabce96ed5b4f11d221 for https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status
2021-01-13 17:47:20,205 tenable.io.exports.ExportsIterator DEBUG EXPORT assets b7c406f7-68ac-4aa5-93f6-52074f43aef9 is status PROCESSING
2021-01-13 17:47:28,211 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status", "params": {}, "body": {}}
2021-01-13 17:47:28,386 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status HTTP/1.1" 200 None
2021-01-13 17:47:28,386 tenable.io.TenableIO DEBUG Request-UUID 15c7e3cb2e86e0eca33fca9f20974870 for https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status
2021-01-13 17:47:28,386 tenable.io.exports.ExportsIterator DEBUG EXPORT assets b7c406f7-68ac-4aa5-93f6-52074f43aef9 is status FINISHED
2021-01-13 17:47:28,386 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/chunks/1", "params": {}, "body": {}}
2021-01-13 17:47:29,739 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/chunks/1 HTTP/1.1" 200 None
2021-01-13 17:47:29,971 tenable.io.TenableIO DEBUG Request-UUID 682529b41459dd9258484a3b77affad0 for https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/chunks/1
2021-01-13 17:47:30,032 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status", "params": {}, "body": {}}
2021-01-13 17:47:30,777 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status HTTP/1.1" 200 None
2021-01-13 17:47:30,778 tenable.io.TenableIO DEBUG Request-UUID 022de3847d9ec3d164137a723d70c58c for https://cloud.tenable.com/assets/export/b7c406f7-68ac-4aa5-93f6-52074f43aef9/status
2021-01-13 17:47:30,778 tenable.io.exports.ExportsIterator DEBUG EXPORT assets b7c406f7-68ac-4aa5-93f6-52074f43aef9 is status FINISHED
2021-01-13 17:47:30,778 tenable.io.TenableIO DEBUG since=None
2021-01-13 17:47:30,778 tenable.io.TenableIO DEBUG first_found=None
2021-01-13 17:47:30,778 tenable.io.TenableIO DEBUG last_found=1610236800
2021-01-13 17:47:30,778 tenable.io.TenableIO DEBUG last_fixed=None
2021-01-13 17:47:30,778 tenable.io.TenableIO DEBUG first_scan_time=None
2021-01-13 17:47:30,778 tenable.io.TenableIO DEBUG last_authenticated_scan_time=None
2021-01-13 17:47:30,778 tenable.io.TenableIO DEBUG last_assessed=None
2021-01-13 17:47:30,778 tenable.io.TenableIO DEBUG {"method": "POST", "url": "https://cloud.tenable.com/vulns/export", "params": {}, "body": {"filters": {"last_found": 1610236800, "severity": ["high", "critical"]}, "num_assets": "1000", "include_unlicensed": true}}
2021-01-13 17:47:32,045 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "POST /vulns/export HTTP/1.1" 200 None
2021-01-13 17:47:32,046 tenable.io.TenableIO DEBUG Request-UUID de2a0f8d7cd6b67c1beaad548246ac66 for https://cloud.tenable.com/vulns/export
2021-01-13 17:47:32,046 tenable.io.TenableIO DEBUG Initiated vuln export 387a1da0-8e33-4bc7-9d8a-37b9afb17bcb
2021-01-13 17:47:32,046 tenable_jira.transform.Tio2Jira INFO Updating and creating issues marked as Open
2021-01-13 17:47:32,046 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/status", "params": {}, "body": {}}
2021-01-13 17:47:33,193 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/status HTTP/1.1" 200 None
2021-01-13 17:47:33,194 tenable.io.TenableIO DEBUG Request-UUID 3ddfd15a546c5a4e630f678179beb06f for https://cloud.tenable.com/vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/status
2021-01-13 17:47:33,194 tenable.io.exports.ExportsIterator DEBUG EXPORT vulns 387a1da0-8e33-4bc7-9d8a-37b9afb17bcb is status PROCESSING
2021-01-13 17:47:35,196 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/status", "params": {}, "body": {}}
2021-01-13 17:47:35,358 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/status HTTP/1.1" 200 None
2021-01-13 17:47:35,359 tenable.io.TenableIO DEBUG Request-UUID 8f4a83efb5b5906f40a2b4a5b3a749c4 for https://cloud.tenable.com/vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/status
2021-01-13 17:47:35,359 tenable.io.exports.ExportsIterator DEBUG EXPORT vulns 387a1da0-8e33-4bc7-9d8a-37b9afb17bcb is status PROCESSING
2021-01-13 17:47:38,362 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/status", "params": {}, "body": {}}
2021-01-13 17:47:38,526 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/status HTTP/1.1" 200 None
2021-01-13 17:47:38,526 tenable.io.TenableIO DEBUG Request-UUID 83b564aab3ea90f63d0b9e2c1aa0e1da for https://cloud.tenable.com/vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/status
2021-01-13 17:47:38,526 tenable.io.exports.ExportsIterator DEBUG EXPORT vulns 387a1da0-8e33-4bc7-9d8a-37b9afb17bcb is status PROCESSING
2021-01-13 17:47:42,531 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/status", "params": {}, "body": {}}
2021-01-13 17:47:42,740 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/status HTTP/1.1" 200 None
2021-01-13 17:47:42,740 tenable.io.TenableIO DEBUG Request-UUID 283937b798aa9ef16627ab4bc603388d for https://cloud.tenable.com/vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/status
2021-01-13 17:47:42,741 tenable.io.exports.ExportsIterator DEBUG EXPORT vulns 387a1da0-8e33-4bc7-9d8a-37b9afb17bcb is status PROCESSING
2021-01-13 17:47:47,746 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/status", "params": {}, "body": {}}
2021-01-13 17:47:47,911 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/status HTTP/1.1" 200 None
2021-01-13 17:47:47,912 tenable.io.TenableIO DEBUG Request-UUID 3912d86a4341a39af7dbae021a2965f2 for https://cloud.tenable.com/vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/status
2021-01-13 17:47:47,912 tenable.io.exports.ExportsIterator DEBUG EXPORT vulns 387a1da0-8e33-4bc7-9d8a-37b9afb17bcb is status PROCESSING
2021-01-13 17:47:47,912 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/chunks/4", "params": {}, "body": {}}
2021-01-13 17:47:49,297 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/chunks/4 HTTP/1.1" 200 1563812
2021-01-13 17:47:49,586 tenable.io.TenableIO DEBUG Request-UUID 09c77b5dc702420c79a2f17ba542a5b9 for https://cloud.tenable.com/vulns/export/387a1da0-8e33-4bc7-9d8a-37b9afb17bcb/chunks/4
2021-01-13 17:47:49,605 root ERROR Caught the following Exception
Traceback (most recent call last):
  File "/home/ubuntu/tenable-jira-cloud-integration/tenable_jira/cli.py", line 151, in cli
    ingest.ingest(observed_since)
  File "/home/ubuntu/tenable-jira-cloud-integration/tenable_jira/transform.py", line 623, in ingest
    self.create_issues(vulns)
  File "/home/ubuntu/tenable-jira-cloud-integration/tenable_jira/transform.py", line 489, in create_issues
    self._process_open_vuln(v, fid)
  File "/home/ubuntu/tenable-jira-cloud-integration/tenable_jira/transform.py", line 360, in _process_open_vuln
    issue, subissue, jql, sjql = self._process_vuln(vuln, fid)
  File "/home/ubuntu/tenable-jira-cloud-integration/tenable_jira/transform.py", line 219, in _process_vuln
    issue = self._gen_issue_skel()
  File "/home/ubuntu/tenable-jira-cloud-integration/tenable_jira/transform.py", line 129, in _gen_issue_skel
    'project': {'key': self._project['key']},
KeyError: 'key'

Available IssueTypes

5: Sub-task
3: Task
10506: Task
10527: Subtask
10525: Task
10536: Task
10538: Subtask
10556: Task
10558: Subtask
patrickoneiltc commented 3 years ago

Note: In this, the custom fields exist. Prior to this, I've deleted everything related to the project before running again. It takes close to half an hour, mostly accounting to the fact that Atlassian is allergic to "bulk action" in the settings.

SteveMcGrath commented 3 years ago

So a couple of things:

The setup-only output is the config file that you would then use for subsequent runs of the integration. It's only really useful if you dont want to have the code suss itself out.

2021-01-13 17:46:40,870 tenable_jira.transform.Tio2Jira INFO Using JIRA Screens []

This tells me that the screens were not selected properly, which would make sense as the project wasnt created, or the user doesnt have permissions to access it.

Whats really weird is that there is a 404 for the project (if it doesn't exist, that makes sense) however no post after it...

2021-01-13 17:46:40,481 urllib3.connectionpool DEBUG https://<JIRA_CLOUD_HOST>:443 "GET /rest/api/3/project/VULN HTTP/1.1" 404 None

Technically speaking it should have attempted to create the project. The code doesn't offer any other option in this regard... https://github.com/tenable/integration-jira-cloud/blob/master/tenable_jira/jira/projects.py#L30-L35. What you should see is something like this:

2021-01-13 14:00:56,417 urllib3.connectionpool DEBUG https://XXXX.atlassian.net:443 "GET /rest/api/3/project/VMNGT6 HTTP/1.1" 404 None
2021-01-13 14:00:56,419 restfly.errors.NotFoundError ERROR [404: GET] https://XXXX.atlassian.net/rest/api/3/project/VMNGT6 body=b'{"errorMessages":["No project could be found with key \'VMNGT6\'."],"errors":{}}'
2021-01-13 14:00:56,419 tenable_jira.jira.Jira INFO Creating Project VMNGT6
2021-01-13 14:00:56,419 tenable_jira.jira.Jira DEBUG Request:{"method": "POST", "url": "https://XXXXX.atlassian.net/rest/api/3/project", "params": {}, "body": {"key": "VMNGT6", "name": "Vuln Management 1234", "projectTypeKey": "business", "projectTemplateKey": "com.atlassian.jira-core-project-templates:jira-core-simplified-task-tracking", "description": "Managing vulnerabilities discovered from Tenable products.", "url": "https://tenable.com", "assigneeType": "UNASSIGNED", "leadAccountId": "XXXXXXXXXX"}}

Truely bizarre.

patrickoneiltc commented 3 years ago

As for the --setup-only, I guess I misread the documentation. I tried different admin levels for the user, and even my own account (jira-admin) and nothing. The script uses the same user that worked previously. I've tried changing the key to one never used before, same error. Troubleshooting is becoming increasingly difficult. About 70% of my attempts to run this script fail because Tenable returns 429 errors on "waiting for existing export." and locks me out for hours. I now have a stable of 4 different API key pairs to use just to be able to run the script to iterate through troubleshooting steps. It's exhausting. Is there any way to repeatedly run the script and not hit these export errors? I.E., can all of the jira work be done first, before Tenable's API is even called?

The only thing we're doing different from the original script is using "requests" instead of "restfly". We still, as of this morning, cannot get a single request through to Jira using "restfly" without an immediate 429 error reply. Atlassian's only reply to this is that we should stop sending so many requests, and when I brought up that it throws this error on the first attempt by a new account, they stopped replying.

Any ideas on next steps in troubleshooting? I am at my wit's end.

(All of this started when I tried to replace a VULN as a business project with VULN as a Service Desk project, as SD supports SLA reporting, which is pretty important to my VM program. The documentation doesn't make clear that other projects won't work, and I was only able to figure that fact out when I went searching for errors and found a note in an issue raised that says only the one business project template is supported. Might I suggest that you update the documentation to call this out?)

SteveMcGrath commented 3 years ago

The "waiting for existing export" issue is a Tenable.io platform issue telling you to slow down on the exports.

As for other project types and not working, I cannot confirm if thats the issue or not, as I have only tested, and only have the ability to test with Business project types and templates. I'm not seeing anything in their docs that would specify differently, however.

As for the RESTfly/429 issue, this problem does appear to be isolated to you at this point, no one else has reported an issue, and I can't re-create the problem on my end. The ONLY difference between what your doing with requests and what RESTfly is doing is that your requests call is making a call without any session handler, whereas RESTfly is using a localized session handler and making calls through that. If that was the core of the issue, then it should have surfaced quite early in development due to the sheer number of calls this thing has to make to tie everything together and keep it in sync on the Jira end. To investigate this one specifically, I'd need more information from Atlassian than "stop making so many calls". One would hope that they have the ability to track calls by user and then relay back what potentially is the issue. The debug logs within the library return back whats actually being sent to their API. It's just really odd that there are a lot of other folks using this without issue.

As for the documentation, I did notice that the type of project that its creating was missing, and will add that to the README doc.

patrickoneiltc commented 3 years ago

As for Restfly? I was only pointing out the only difference I could think of. Before I tried to rebuild VULN, requests worked without issue. It handled transactions for a couple of weeks. I really only pointed it out because, other than a swap from Restfly to Requests, we've made no other alterations to the code.

The export errors from Tenable can happen for several hours after a failed attempt. The absolute fastest I've been able to use the same API key after a failure is 15 minutes. This is making it incredibly difficult to troubleshoot the issues causing the failure.

Back to the issue at hand, ["No project could be found with key 'VULN'."], Is there any troubleshooting steps you can think of? I have devoted most of tomorrow to getting this up and running.

SteveMcGrath commented 3 years ago

are you manually creating the project or letting the script handle it? I did a little investigating into the Jira API before my response earlier this morning, and I'm wonder if if this may work for you for the jira project config section:

project:
  key: VMNGT
  name: Vuln Management
  projectTypeKey: service_desk
  projectTemplateKey: com.atlassian.servicedesk:simplified-it-service-desk
  leadAccountId: XXXX

It'll create a new project using the VMNGT key (instead of the VULN one) and use the service_desk type. You may need to specify the screens manually in the config after project creation, which you will know if you see the following in the log:

2021-01-13 17:46:40,870 tenable_jira.transform.Tio2Jira INFO Using JIRA Screens []

For testing purposes, if you have a scan that has completed recently, you can always reign in the export by passing the following as part of the tenable stanza, note that the integer value is the number of calendar days to go back:

tenable:
  tio_age: 7

The shorter the time, typically the quicker the export. Once you do get it working, I would highly recommend running this as s service by setting the service stanza and then letting the integration handle it's own temporal tracking. In this case the integer is the number of hours between exports:

service:
  interval: 24

Again as noted earlier, the initial 404 from the GET call is normal if the project doesn't exist, however it should try to create it with a POST call as the next call.

Sadly I cant offer anything further with the 429s on the Atlassian side. I'm sorry your running into all these issues, however I'd have to do some really deep digging, likely on your setup, in order to root out what specifically is happening there. If it comes to that, it may require reaching out to Professional Services.

patrickoneiltc commented 3 years ago

Well now I know that the error message is not project specific. I got what seems to be the same error for your above service desk project vs the standard business. To note, the VMNGT key and Vuln Management name have never before been used in our jira. Output below.

I have tried both letting the script create the project, and creating one ahead of time. I cannot create a project in Jira manually that has the template used in the script, and as such the script fails trying to find a specific screen.

At most, I am running the script with tio_age set to 3. Mostly, I run it with it set to 1. I do not think that the export errors I'm experiencing have anything to do with volume at all, as we have a very small network and 1-3 days of data should be trivial to process.

I'm no longer worried about Restfly and 429 errors from Atlassian. I have something that works. It was meatball surgery, but it's what we have time for here. I work for a very small company with a very small engineering team, and "good enough" often is the only solution available to us. As long as requests works, I will not be pursuing any further troubleshooting for Restfly.

When the project was working, we did run it as a service in the background with the interval set to either 12 or 24.

LOOK HERE: {'errorMessages': ["No project could be found with key 'VMNGT'."], 'errors': {}}

Configuration File:

jira:
  address: <REDACTED>
  api_token: <REDACTED>
  api_username: <REDACTED>
log:
  filename: /var/log/tenable.log
  format: '%(asctime)-15s %(name)s %(levelname)s %(message)s'
  level: 20
project:
  key: VMNGT
  leadAccountId: <REDACTED>
  name: Vuln Management
  projectTemplateKey: com.atlassian.servicedesk:simplified-it-service-desk
  projectTypeKey: service_desk
service:
  interval: 1
tenable:
  access_key: <REDACTED>
  platform: tenable.io
  secret_key: <REDACTED>
  tio_age: 1
  tio_severities:
  - high
  - critical

Debug Logs

2021-01-14 17:44:23,359 root INFO Tenable2JiraCloud Version 1.1.18
2021-01-14 17:44:23,360 root INFO Using configuration file config.yaml
2021-01-14 17:44:23,360 root INFO Running on Python 3.6.9 Linux/x86_64
2021-01-14 17:44:23,360 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/session", "params": {}, "body": {}}
2021-01-14 17:44:23,363 urllib3.connectionpool DEBUG Starting new HTTPS connection (1): cloud.tenable.com:443
2021-01-14 17:44:23,641 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /session HTTP/1.1" 200 None
2021-01-14 17:44:23,643 tenable.io.TenableIO DEBUG Request-UUID 026dc0b6d5b346617db8446a23fcd48d for https://cloud.tenable.com/session
2021-01-14 17:44:23,644 urllib3.connectionpool DEBUG Starting new HTTPS connection (1): <JIRA_CLOUD_HOST>:443
2021-01-14 17:44:23,724 urllib3.connectionpool DEBUG https://<JIRA_CLOUD_HOST>:443 "GET /rest/api/3/project/VMNGT HTTP/1.1" 404 None
2021-01-14 17:44:23,726 urllib3.connectionpool DEBUG Starting new HTTPS connection (1): <JIRA_CLOUD_HOST>:443
2021-01-14 17:44:23,881 urllib3.connectionpool DEBUG https://<JIRA_CLOUD_HOST>:443 "GET /rest/api/3/field HTTP/1.1" 200 None
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Tenable Platform is customfield_11593 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO CVEs is customfield_11594 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO CVSSv2 Base Score is customfield_11595 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO CVSSv2 Temporal Score is customfield_11596 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO CVSSv3 Base Score is customfield_11597 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO CVSSv3 Temporal Score is customfield_11598 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Patch Publication Date is customfield_11599 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Tenable Plugin ID is customfield_11600 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Tenable Plugin Family is customfield_11601 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Tenable Plugin Name is customfield_11602 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Vulnerability Severity is customfield_11603 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Tenable Asset UUID is customfield_11604 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Tenable Asset Tags is customfield_11605 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Device MAC Addresses is customfield_11606 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Device IPv4 Addresses is customfield_11607 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Device IPv6 Addresses is customfield_11608 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Device Hostname is customfield_11609 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Device NetBIOS Name is customfield_11610 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Device DNS Name is customfield_11611 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Device Network ID is customfield_11612 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Vulnerability First Seen is customfield_11613 (existing)
2021-01-14 17:44:23,886 tenable_jira.jira.Jira INFO Vulnerability Last Seen is customfield_11614 (existing)
2021-01-14 17:44:23,887 tenable_jira.jira.Jira INFO Vulnerability Last Fixed is customfield_11615 (existing)
2021-01-14 17:44:23,887 tenable_jira.jira.Jira INFO Vulnerability State is customfield_11616 (existing)
2021-01-14 17:44:23,887 tenable_jira.jira.Jira INFO Vulnerability Port is customfield_11617 (existing)
2021-01-14 17:44:23,887 tenable_jira.jira.Jira INFO Vulnerability Protocol is customfield_11618 (existing)
2021-01-14 17:44:23,887 tenable_jira.jira.Jira INFO Vulnerability Repository ID is customfield_11619 (existing)
2021-01-14 17:44:23,887 tenable_jira.jira.Jira INFO Vulnerability Repository Name is customfield_11620 (existing)
2021-01-14 17:44:23,887 tenable_jira.jira.Jira INFO Tenable VPR Score is customfield_11621 (existing)
2021-01-14 17:44:23,888 urllib3.connectionpool DEBUG Starting new HTTPS connection (1): <JIRA_CLOUD_HOST>:443
2021-01-14 17:44:23,962 urllib3.connectionpool DEBUG https://<JIRA_CLOUD_HOST>:443 "GET /rest/api/3/issuetype HTTP/1.1" 200 None
2021-01-14 17:44:23,966 urllib3.connectionpool DEBUG Starting new HTTPS connection (1): <JIRA_CLOUD_HOST>:443
2021-01-14 17:44:24,021 urllib3.connectionpool DEBUG https://<JIRA_CLOUD_HOST>:443 "GET /rest/api/3/screens?startAt=-100&maxResults=100 HTTP/1.1" 200 None
2021-01-14 17:44:24,026 urllib3.connectionpool DEBUG Starting new HTTPS connection (1): <JIRA_CLOUD_HOST>:443
2021-01-14 17:44:24,102 urllib3.connectionpool DEBUG https://<JIRA_CLOUD_HOST>:443 "GET /rest/api/3/screens?startAt=0&maxResults=100 HTTP/1.1" 200 None
2021-01-14 17:44:24,106 tenable_jira.transform.Tio2Jira INFO Using JIRA Screens []
2021-01-14 17:44:24,107 tenable.io.TenableIO DEBUG {"method": "POST", "url": "https://cloud.tenable.com/assets/export", "params": {}, "body": {"filters": {"updated_at": 1610496000}, "chunk_size": 1000}}
2021-01-14 17:44:24,716 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "POST /assets/export HTTP/1.1" 200 None
2021-01-14 17:44:24,717 tenable.io.TenableIO DEBUG Request-UUID 7b4891009e73220c834ba7a8656a10cc for https://cloud.tenable.com/assets/export
2021-01-14 17:44:24,717 tenable.io.TenableIO DEBUG Initiated asset export dbdb4338-a674-4525-8814-c27ac31664b2
2021-01-14 17:44:24,717 tenable.io.TenableIO DEBUG {"method": "POST", "url": "https://cloud.tenable.com/assets/export", "params": {}, "body": {"filters": {"deleted_at": 1610496000}, "chunk_size": 1000}}
2021-01-14 17:44:25,608 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "POST /assets/export HTTP/1.1" 200 None
2021-01-14 17:44:25,609 tenable.io.TenableIO DEBUG Request-UUID 6bbee479962bce647c2bb265ca15052e for https://cloud.tenable.com/assets/export
2021-01-14 17:44:25,609 tenable.io.TenableIO DEBUG Initiated asset export 90278d53-a56f-4f78-a4fd-ac2d1a24595a
2021-01-14 17:44:25,609 tenable.io.TenableIO DEBUG {"method": "POST", "url": "https://cloud.tenable.com/assets/export", "params": {}, "body": {"filters": {"terminated_at": 1610496000}, "chunk_size": 1000}}
2021-01-14 17:44:26,705 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "POST /assets/export HTTP/1.1" 200 None
2021-01-14 17:44:26,706 tenable.io.TenableIO DEBUG Request-UUID a4ce6dd36f2f4e2c73932fe916756b9d for https://cloud.tenable.com/assets/export
2021-01-14 17:44:26,706 tenable.io.TenableIO DEBUG Initiated asset export 139902ac-75ea-40e2-93c0-3ef0752c4ebd
2021-01-14 17:44:26,706 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/139902ac-75ea-40e2-93c0-3ef0752c4ebd/status", "params": {}, "body": {}}
2021-01-14 17:44:27,902 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/139902ac-75ea-40e2-93c0-3ef0752c4ebd/status HTTP/1.1" 200 None
2021-01-14 17:44:27,903 tenable.io.TenableIO DEBUG Request-UUID c34e277c9e707dac71f01f0307aba79d for https://cloud.tenable.com/assets/export/139902ac-75ea-40e2-93c0-3ef0752c4ebd/status
2021-01-14 17:44:27,903 tenable.io.exports.ExportsIterator DEBUG EXPORT assets 139902ac-75ea-40e2-93c0-3ef0752c4ebd is status FINISHED
2021-01-14 17:44:27,903 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/139902ac-75ea-40e2-93c0-3ef0752c4ebd/chunks/1", "params": {}, "body": {}}
2021-01-14 17:44:29,092 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/139902ac-75ea-40e2-93c0-3ef0752c4ebd/chunks/1 HTTP/1.1" 200 None
2021-01-14 17:44:29,093 tenable.io.TenableIO DEBUG Request-UUID a0354b3a87fe635fee50bc646557c22e for https://cloud.tenable.com/assets/export/139902ac-75ea-40e2-93c0-3ef0752c4ebd/chunks/1
2021-01-14 17:44:29,094 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/139902ac-75ea-40e2-93c0-3ef0752c4ebd/status", "params": {}, "body": {}}
2021-01-14 17:44:30,304 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/139902ac-75ea-40e2-93c0-3ef0752c4ebd/status HTTP/1.1" 200 None
2021-01-14 17:44:30,305 tenable.io.TenableIO DEBUG Request-UUID 399412f88ffbd32bc795142935c84207 for https://cloud.tenable.com/assets/export/139902ac-75ea-40e2-93c0-3ef0752c4ebd/status
2021-01-14 17:44:30,305 tenable.io.exports.ExportsIterator DEBUG EXPORT assets 139902ac-75ea-40e2-93c0-3ef0752c4ebd is status FINISHED
2021-01-14 17:44:30,305 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/90278d53-a56f-4f78-a4fd-ac2d1a24595a/status", "params": {}, "body": {}}
2021-01-14 17:44:31,577 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/90278d53-a56f-4f78-a4fd-ac2d1a24595a/status HTTP/1.1" 200 None
2021-01-14 17:44:31,578 tenable.io.TenableIO DEBUG Request-UUID 3b8453618b07463a841a13b2534de653 for https://cloud.tenable.com/assets/export/90278d53-a56f-4f78-a4fd-ac2d1a24595a/status
2021-01-14 17:44:31,578 tenable.io.exports.ExportsIterator DEBUG EXPORT assets 90278d53-a56f-4f78-a4fd-ac2d1a24595a is status FINISHED
2021-01-14 17:44:31,578 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/dbdb4338-a674-4525-8814-c27ac31664b2/status", "params": {}, "body": {}}
2021-01-14 17:44:32,733 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/dbdb4338-a674-4525-8814-c27ac31664b2/status HTTP/1.1" 200 None
2021-01-14 17:44:32,734 tenable.io.TenableIO DEBUG Request-UUID bec7ea4e30e6388ac08b2d987de9a44a for https://cloud.tenable.com/assets/export/dbdb4338-a674-4525-8814-c27ac31664b2/status
2021-01-14 17:44:32,734 tenable.io.exports.ExportsIterator DEBUG EXPORT assets dbdb4338-a674-4525-8814-c27ac31664b2 is status FINISHED
2021-01-14 17:44:32,734 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/dbdb4338-a674-4525-8814-c27ac31664b2/chunks/1", "params": {}, "body": {}}
2021-01-14 17:44:34,045 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/dbdb4338-a674-4525-8814-c27ac31664b2/chunks/1 HTTP/1.1" 200 None
2021-01-14 17:44:34,276 tenable.io.TenableIO DEBUG Request-UUID 6cafa8d51ec58ae602b4311b0a7a871a for https://cloud.tenable.com/assets/export/dbdb4338-a674-4525-8814-c27ac31664b2/chunks/1
2021-01-14 17:44:34,337 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/assets/export/dbdb4338-a674-4525-8814-c27ac31664b2/status", "params": {}, "body": {}}
2021-01-14 17:44:35,336 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /assets/export/dbdb4338-a674-4525-8814-c27ac31664b2/status HTTP/1.1" 200 None
2021-01-14 17:44:35,337 tenable.io.TenableIO DEBUG Request-UUID 2889b456cc9464e9b5aca6810161be3d for https://cloud.tenable.com/assets/export/dbdb4338-a674-4525-8814-c27ac31664b2/status
2021-01-14 17:44:35,337 tenable.io.exports.ExportsIterator DEBUG EXPORT assets dbdb4338-a674-4525-8814-c27ac31664b2 is status FINISHED
2021-01-14 17:44:35,337 tenable.io.TenableIO DEBUG since=None
2021-01-14 17:44:35,337 tenable.io.TenableIO DEBUG first_found=None
2021-01-14 17:44:35,337 tenable.io.TenableIO DEBUG last_found=1610496000
2021-01-14 17:44:35,337 tenable.io.TenableIO DEBUG last_fixed=None
2021-01-14 17:44:35,337 tenable.io.TenableIO DEBUG first_scan_time=None
2021-01-14 17:44:35,337 tenable.io.TenableIO DEBUG last_authenticated_scan_time=None
2021-01-14 17:44:35,337 tenable.io.TenableIO DEBUG last_assessed=None
2021-01-14 17:44:35,337 tenable.io.TenableIO DEBUG {"method": "POST", "url": "https://cloud.tenable.com/vulns/export", "params": {}, "body": {"filters": {"last_found": 1610496000, "severity": ["high", "critical"]}, "num_assets": "1000", "include_unlicensed": true}}
2021-01-14 17:44:36,355 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "POST /vulns/export HTTP/1.1" 200 None
2021-01-14 17:44:36,356 tenable.io.TenableIO DEBUG Request-UUID 8c7f5413c571b9187468d824ef27071b for https://cloud.tenable.com/vulns/export
2021-01-14 17:44:36,356 tenable.io.TenableIO DEBUG Initiated vuln export bcec8e4a-6b31-481d-8924-cd0f95f8bbc5
2021-01-14 17:44:36,356 tenable_jira.transform.Tio2Jira INFO Updating and creating issues marked as Open
2021-01-14 17:44:36,356 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/vulns/export/bcec8e4a-6b31-481d-8924-cd0f95f8bbc5/status", "params": {}, "body": {}}
2021-01-14 17:44:37,462 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /vulns/export/bcec8e4a-6b31-481d-8924-cd0f95f8bbc5/status HTTP/1.1" 200 None
2021-01-14 17:44:37,463 tenable.io.TenableIO DEBUG Request-UUID 7645a49eef0552e9a1c415abb8561559 for https://cloud.tenable.com/vulns/export/bcec8e4a-6b31-481d-8924-cd0f95f8bbc5/status
2021-01-14 17:44:37,463 tenable.io.exports.ExportsIterator DEBUG EXPORT vulns bcec8e4a-6b31-481d-8924-cd0f95f8bbc5 is status PROCESSING
2021-01-14 17:44:39,465 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/vulns/export/bcec8e4a-6b31-481d-8924-cd0f95f8bbc5/status", "params": {}, "body": {}}
2021-01-14 17:44:39,618 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /vulns/export/bcec8e4a-6b31-481d-8924-cd0f95f8bbc5/status HTTP/1.1" 200 None
2021-01-14 17:44:39,619 tenable.io.TenableIO DEBUG Request-UUID f31f45e4493f8f818e9e30221833dcdf for https://cloud.tenable.com/vulns/export/bcec8e4a-6b31-481d-8924-cd0f95f8bbc5/status
2021-01-14 17:44:39,619 tenable.io.exports.ExportsIterator DEBUG EXPORT vulns bcec8e4a-6b31-481d-8924-cd0f95f8bbc5 is status PROCESSING
2021-01-14 17:44:42,622 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/vulns/export/bcec8e4a-6b31-481d-8924-cd0f95f8bbc5/status", "params": {}, "body": {}}
2021-01-14 17:44:42,864 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /vulns/export/bcec8e4a-6b31-481d-8924-cd0f95f8bbc5/status HTTP/1.1" 200 None
2021-01-14 17:44:42,865 tenable.io.TenableIO DEBUG Request-UUID b43d9f844abe7cca7b6bf8e4b1f958f5 for https://cloud.tenable.com/vulns/export/bcec8e4a-6b31-481d-8924-cd0f95f8bbc5/status
2021-01-14 17:44:42,865 tenable.io.exports.ExportsIterator DEBUG EXPORT vulns bcec8e4a-6b31-481d-8924-cd0f95f8bbc5 is status PROCESSING
2021-01-14 17:44:46,870 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/vulns/export/bcec8e4a-6b31-481d-8924-cd0f95f8bbc5/status", "params": {}, "body": {}}
2021-01-14 17:44:47,046 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /vulns/export/bcec8e4a-6b31-481d-8924-cd0f95f8bbc5/status HTTP/1.1" 200 None
2021-01-14 17:44:47,047 tenable.io.TenableIO DEBUG Request-UUID 085fe992389015a5b425ab5223a41c58 for https://cloud.tenable.com/vulns/export/bcec8e4a-6b31-481d-8924-cd0f95f8bbc5/status
2021-01-14 17:44:47,047 tenable.io.exports.ExportsIterator DEBUG EXPORT vulns bcec8e4a-6b31-481d-8924-cd0f95f8bbc5 is status PROCESSING
2021-01-14 17:44:47,047 tenable.io.TenableIO DEBUG {"method": "GET", "url": "https://cloud.tenable.com/vulns/export/bcec8e4a-6b31-481d-8924-cd0f95f8bbc5/chunks/2", "params": {}, "body": {}}
2021-01-14 17:44:48,373 urllib3.connectionpool DEBUG https://cloud.tenable.com:443 "GET /vulns/export/bcec8e4a-6b31-481d-8924-cd0f95f8bbc5/chunks/2 HTTP/1.1" 200 283639
2021-01-14 17:44:48,537 tenable.io.TenableIO DEBUG Request-UUID fdd361cbb361dcca9715ea2696e08501 for https://cloud.tenable.com/vulns/export/bcec8e4a-6b31-481d-8924-cd0f95f8bbc5/chunks/2
2021-01-14 17:44:48,542 root ERROR Caught the following Exception
Traceback (most recent call last):
  File "/home/ubuntu/tenable-jira-cloud-integration/tenable_jira/cli.py", line 151, in cli
    ingest.ingest(observed_since)
  File "/home/ubuntu/tenable-jira-cloud-integration/tenable_jira/transform.py", line 623, in ingest
    self.create_issues(vulns)
  File "/home/ubuntu/tenable-jira-cloud-integration/tenable_jira/transform.py", line 489, in create_issues
    self._process_open_vuln(v, fid)
  File "/home/ubuntu/tenable-jira-cloud-integration/tenable_jira/transform.py", line 360, in _process_open_vuln
    issue, subissue, jql, sjql = self._process_vuln(vuln, fid)
  File "/home/ubuntu/tenable-jira-cloud-integration/tenable_jira/transform.py", line 219, in _process_vuln
    issue = self._gen_issue_skel()
  File "/home/ubuntu/tenable-jira-cloud-integration/tenable_jira/transform.py", line 129, in _gen_issue_skel
    'project': {'key': self._project['key']},
KeyError: 'key'

Available IssueTypes

5: Sub-task
3: Task
10506: Task
10527: Subtask
10525: Task
10536: Task
10538: Subtask
10556: Task
10558: Subtask
/-------------------------------NOTICE-----------------------------------\ The output above is helpful for us to troubleshoot exactly what is happening within the code and offer a diagnosis for how to correct. Please note that while some basic redaction has already been performed that we ask you to review the information you're about to send and ensure that nothing deemed sensitive is transmitted.
-- Copy of output saved to "issue_debug.md"

------------------------------------------------------------------------/

SteveMcGrath commented 3 years ago

Alrighty, lets see what happens here:

#!/usr/bin/env python
import yaml, json, logging
from tenable_jira.config import base_config
from tenable.utils import dict_merge
from tenable.io import TenableIO
from tenable_jira.jira import Jira

logging.basicConfig(level=logging.DEBUG)

config = dict_merge(base_config(), yaml.load(open('config.yaml'), Loader=yaml.Loader))

jira = Jira(
    'https://{}/rest/api/3'.format(config['jira']['address']),
    config['jira']['api_username'],
    config['jira']['api_token']
)

print(json.dumps(config['project'], sort_keys=True, indent=4))

p = jira.projects.upsert(**config['project'])

print(p)
SteveMcGrath commented 3 years ago

whats odd is the lack of the post afterwards. Lets try to manually create here:

#!/usr/bin/env python
import yaml, json, logging
from tenable_jira.config import base_config
from tenable.utils import dict_merge
from tenable.io import TenableIO
from tenable_jira.jira import Jira

logging.basicConfig(level=logging.DEBUG)

config = dict_merge(base_config(), yaml.load(open('config.yaml'), Loader=yaml.Loader))

jira = Jira(
    'https://{}/rest/api/3'.format(config['jira']['address']),
    config['jira']['api_username'],
    config['jira']['api_token']
)

print(json.dumps(config['project'], sort_keys=True, indent=4))

p = jira.projects.create(**config['project'])

print(p)
patrickoneiltc commented 3 years ago

{ "assigneeType": "UNASSIGNED", "description": "Managing vulnerabilities discovered from Tenable products.", "key": "VMNGT", "leadAccountId": "[REDACTED]", "name": "Vuln Management", "projectTemplateKey": "com.atlassian.servicedesk:simplified-it-service-desk", "projectTypeKey": "service_desk", "url": "https://tenable.com" } DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1):[REDACTED].atlassian.net:443 DEBUG:urllib3.connectionpool:https://[REDACTED].atlassian.net:443 "GET /rest/api/3/project/VMNGT HTTP/1.1" 404 None {'errorMessages': ["No project could be found with key 'VMNGT'."], 'errors': {}}

patrickoneiltc commented 3 years ago

Had to delete and repost due to missing a redaction.

{ "assigneeType": "UNASSIGNED", "description": "Managing vulnerabilities discovered from Tenable products.", "key": "VMNGT", "leadAccountId": "[REDACTED]", "name": "Vuln Management", "projectTemplateKey": "com.atlassian.servicedesk:simplified-it-service-desk", "projectTypeKey": "service_desk", "url": "https://tenable.com" } DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): [REDACTED].atlassian.net:443 DEBUG:urllib3.connectionpool:https://[REDACTED].atlassian.net:443 "GET /rest/api/3/project/VMNGT HTTP/1.1" 404 None {'errorMessages': ["No project could be found with key 'VMNGT'."], 'errors': {}}

SteveMcGrath commented 3 years ago

well thats unfortunate, according to their API docs, that template should work.

https://developer.atlassian.com/cloud/jira/platform/rest/v3/api-group-projects/#api-rest-api-3-project-post

Try this template instead: com.atlassian.servicedesk:simplified-internal-service-desk

SteveMcGrath commented 3 years ago

Or, you could try to create it manually as the VMNGT key

patrickoneiltc commented 3 years ago

{ "assigneeType": "UNASSIGNED", "description": "Managing vulnerabilities discovered from Tenable products.", "key": "VMNGT", "leadAccountId": "REDACTED", "name": "Vuln Management", "projectTemplateKey": "com.atlassian.servicedesk:simplified-internal-service-desk", "projectTypeKey": "service_desk", "url": "https://tenable.com" } DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): REDACTED.atlassian.net:443 DEBUG:urllib3.connectionpool:https://REDACTED.atlassian.net:443 "POST /rest/api/3/project HTTP/1.1" 201 None {'self': 'https://REDACTED.atlassian.net/rest/api/3/project/12756', 'id': 12756, 'key': 'VMNGT'}

SteveMcGrath commented 3 years ago

well that worked. Ok, so lets try to run it using --setup-only. This way it'll only do the creation and tie-up of the Jira stuff.

patrickoneiltc commented 3 years ago
closed_transitions:
- Closed
- Done
- Resolved
fields:
- is_platform_id: true
  issue_type:
  - Sub-Task
  jira_field: Tenable Platform
  jira_id: customfield_11593
  searcher: textsearcher
  type: readonlyfield
- issue_type:
  - Task
  jira_field: CVEs
  jira_id: customfield_11594
  searcher: labelsearcher
  tio_field: plugin.cve
  tsc_field: cve
  type: labels
- issue_type:
  - Task
  - Sub-task
  jira_field: CVSSv2 Base Score
  jira_id: customfield_11595
  searcher: textsearcher
  tio_field: plugin.cvss_base_score
  tsc_field: baseScore
  type: readonlyfield
- issue_type:
  - Task
  - Sub-task
  jira_field: CVSSv2 Temporal Score
  jira_id: customfield_11596
  searcher: textsearcher
  tio_field: plugin.cvss_temporal_score
  tsc_field: temporalScore
  type: readonlyfield
- issue_type:
  - Task
  - Sub-task
  jira_field: CVSSv3 Base Score
  jira_id: customfield_11597
  searcher: textsearcher
  tio_field: plugin.cvss3_base_score
  tsc_field: cvssV3BaseScore
  type: readonlyfield
- issue_type:
  - Task
  - Sub-task
  jira_field: CVSSv3 Temporal Score
  jira_id: customfield_11598
  searcher: textsearcher
  tio_field: plugin.cvss3_temporal_score
  tsc_field: cvssV3TemporalScore
  type: readonlyfield
- issue_type:
  - Task
  - Sub-task
  jira_field: Patch Publication Date
  jira_id: customfield_11599
  searcher: textsearcher
  tio_field: plugin.patch_publication_date
  tsc_field: patchPubDate
  type: readonlyfield
- issue_type:
  - Task
  - Sub-task
  jira_field: Tenable Plugin ID
  jira_id: customfield_11600
  searcher: textsearcher
  tio_field: plugin.id
  tsc_field: pluginID
  type: readonlyfield
- issue_type:
  - Task
  - Sub-task
  jira_field: Tenable Plugin Family
  jira_id: customfield_11601
  searcher: textsearcher
  tio_field: plugin.family
  tsc_field: family.name
  type: readonlyfield
- issue_type:
  - Task
  - Sub-task
  jira_field: Tenable Plugin Name
  jira_id: customfield_11602
  searcher: textsearcher
  tio_field: plugin.name
  tsc_field: pluginName
  type: readonlyfield
- issue_type:
  - Task
  - Sub-task
  jira_field: Vulnerability Severity
  jira_id: customfield_11603
  searcher: textsearcher
  tio_field: plugin.risk_factor
  tsc_field: severity.name
  type: readonlyfield
- issue_type:
  - Sub-task
  jira_field: Tenable Asset UUID
  jira_id: customfield_11604
  searcher: labelsearcher
  tio_field: asset.uuid
  tsc_field: uuid
  type: labels
- issue_type:
  - Sub-task
  jira_field: Tenable Asset Tags
  jira_id: customfield_11605
  searcher: labelsearcher
  tio_field: asset.tags
  type: labels
- issue_type:
  - Sub-task
  jira_field: Device MAC Addresses
  jira_id: customfield_11606
  searcher: textsearcher
  tio_field: asset.mac_address
  tsc_field: macAddress
  type: readonlyfield
- issue_type:
  - Sub-task
  jira_field: Device IPv4 Addresses
  jira_id: customfield_11607
  searcher: labelsearcher
  tio_field: asset.ipv4
  tsc_field: ip
  type: labels
- issue_type:
  - Sub-task
  jira_field: Device IPv6 Addresses
  jira_id: customfield_11608
  searcher: labelsearcher
  tio_field: asset.ipv6
  type: labels
- issue_type:
  - Sub-task
  jira_field: Device Hostname
  jira_id: customfield_11609
  searcher: labelsearcher
  tio_field: asset.hostname
  tsc_field: dnsName
  type: labels
- issue_type:
  - Sub-task
  jira_field: Device NetBIOS Name
  jira_id: customfield_11610
  searcher: textsearcher
  tsc_field: netbiosName
  type: readonlyfield
- issue_type:
  - Sub-task
  jira_field: Device DNS Name
  jira_id: customfield_11611
  searcher: textsearcher
  tio_field: asset.fqdn
  tsc_field: dnsName
  type: readonlyfield
- issue_type:
  - Sub-task
  jira_field: Device Network ID
  jira_id: customfield_11612
  searcher: textsearcher
  tio_field: asset.network_id
  type: readonlyfield
- issue_type:
  - Sub-task
  jira_field: Vulnerability First Seen
  jira_id: customfield_11613
  searcher: datetimerange
  tio_field: first_found
  tsc_field: firstSeen
  type: datetime
- issue_type:
  - Sub-task
  jira_field: Vulnerability Last Seen
  jira_id: customfield_11614
  searcher: datetimerange
  tio_field: last_found
  tsc_field: lastSeen
  type: datetime
- issue_type:
  - Sub-task
  jira_field: Vulnerability Last Fixed
  jira_id: customfield_11615
  searcher: datetimerange
  tio_field: last_fixed
  type: datetime
- issue_type:
  - Sub-task
  jira_field: Vulnerability State
  jira_id: customfield_11616
  searcher: textsearcher
  tio_field: state
  type: readonlyfield
- issue_type:
  - Sub-task
  jira_field: Vulnerability Port
  jira_id: customfield_11617
  searcher: textsearcher
  tio_field: port.port
  tsc_field: port
  type: readonlyfield
- issue_type:
  - Sub-task
  jira_field: Vulnerability Protocol
  jira_id: customfield_11618
  searcher: textsearcher
  tio_field: port.protocol
  tsc_field: protocol
  type: readonlyfield
- issue_type:
  - Sub-task
  jira_field: Vulnerability Repository ID
  jira_id: customfield_11619
  searcher: textsearcher
  tsc_field: repository.id
  type: readonlyfield
- issue_type:
  - Sub-task
  jira_field: Vulnerability Repository Name
  jira_id: customfield_11620
  searcher: textsearcher
  tsc_field: repository.name
  type: readonlyfield
- issue_type:
  - Task
  - Sub-task
  jira_field: Tenable VPR Score
  jira_id: customfield_11621
  searcher: textsearcher
  tio_field: plugin.vpr.score
  tsc_field: vprScore
  type: readonlyfield
issue_default_fields:
  description:
    Sub-task:
    - name: Description
      tio_field: '{vuln[plugin.description]}'
      tsc_field: '{vuln[description]}'
    - name: Solution
      tio_field: '{vuln[plugin.solution]}'
      tsc_field: '{vuln[solution]}'
    - name: Output
      tio_field: '{vuln[output]}'
      tsc_field: '{vuln[pluginOutput]}'
    Task:
    - name: Description
      tio_field: '{vuln[plugin.description]}'
      tsc_field: '{vuln[description]}'
    - name: Solution
      tio_field: '{vuln[plugin.solution]}'
      tsc_field: '{vuln[solution]}'
  summary:
    Sub-task:
      tio_field: '[{vuln[asset.hostname]}/{vuln[port.port]}/{vuln[port.protocol]}]
        [{vuln[plugin.id]}] {vuln[plugin.name]}'
      tsc_field: '[{vuln[ip]}/{vuln[port]}/{vuln[protocol]}] [{vuln[pluginID]}] {vuln[pluginName]}'
    Task:
      tio_field: '[{vuln[plugin.id]}] {vuln[plugin.name]}'
      tsc_field: '[{vuln[pluginID]}] {vuln[pluginName]}'
issue_types:
- jira_id: '3'
  name: Task
  search:
  - Tenable Plugin ID
  type: standard
- jira_id: '5'
  name: Sub-task
  search:
  - Tenable Platform
  - Tenable Plugin ID
  - Tenable Asset UUID
  - Device IPv4 Addresses
  - Device IPv6 Addresses
  - Vulnerability Port
  - Vulnerability Protocol
  type: subtask
jira:
  address: [redacted].atlassian.net
  api_token: [redacted]
  api_username: [redacted]
log:
  filename: /var/log/tenable.log
  format: '%(asctime)-15s %(name)s %(levelname)s %(message)s'
  level: 20
project:
  assigneeType: UNASSIGNED
  description: Managing vulnerabilities discovered from Tenable products.
  key: VMNGT
  leadAccountId: [redacted]
  name: Vuln Management
  projectTemplateKey: com.atlassian.servicedesk:simplified-internal-service-desk
  projectTypeKey: service_desk
  url: https://tenable.com
screen:
  name:
  - Task Management Edit/View Issue Screen
  - Task Management Create Issue Screen
  no_create: true
  tabs:
    Asset:
    - Tenable Asset UUID
    - Tenable Asset Tags
    - Tenable Platform
    - Device Hostname
    - Device NetBIOS Name
    - Device DNS Name
    - Device IPv4 Addresses
    - Device IPv6 Addresses
    - Device MAC Addresses
    - Device Network ID
    - Vulnerability Repository ID
    - Vulnerability Repository Name
    Vulnerability:
    - CVEs
    - Tenable VPR Score
    - CVSSv2 Base Score
    - CVSSv2 Temporal Score
    - CVSSv3 Base Score
    - CVSSv3 Temporal Score
    - Tenable Plugin ID
    - Tenable Plugin Family
    - Tenable Plugin Name
    - Vulnerability Severity
    - Vulnerability First Seen
    - Vulnerability Last Seen
    - Vulnerability Last Fixed
    - Vulnerability State
    - Vulnerability Port
    - Vulnerability Protocol
    - Patch Publication Date
service:
  interval: 1
tenable:
  access_key: [redacted]
  address: null
  chunk_size: 1000
  page_size: 1000
  password: null
  platform: tenable.io
  port: 443
  query_id: null
  secret_key: [redacted]
  tio_age: 1
  tio_severities:
  - high
  - critical
  username: null
SteveMcGrath commented 3 years ago

Ok, now as I can assume that "Task" and Sub-Task" dont necessarially apply to this project type, you're going to need to manually verify that the issue types are correct. The script selected ID 3 for "Task" and ID 5 for "Sub-Task". You will need to review those. It also appeared to have not selected the screens, likely as it wasn't intended for this project type. You will need to discover the Jira IDs for these screens and supply them manually as well. For information on how to do that, take a look at this comment:

https://github.com/tenable/integration-jira-cloud/issues/23#issuecomment-604631753

For the issuetypes, the only way I have found to get the Ids from their UI is to goto the project settings, then click on the issue types section in the menu, then click the issuetypes that we want to map here and in the URL you should see a integer value.

Screen Shot 2021-01-14 at 2 08 31 PM

patrickoneiltc commented 3 years ago

I'm getting the Tenable 429s again. I haven't run the script in several hours. However, I only have one screen created for the new SD project, and no one who works here has the expertise to create a new one from scratch, so I do not think we can pursue using SD if it requires creating new screens and task type by hand. The random and seemingly arbitrary Tenable 429s have me seeing red. I'll address this again tomorrow.

SteveMcGrath commented 3 years ago

worst case, you now have a pathway to at least get the script to create the project and work from there. If you want to pursue SD further, I would recommend engaging with our pro-serve team as they have a lot of experience writing these kind of things to be specifically tailored to your needs.

patrickoneiltc commented 3 years ago

I could not get the config to read at all when I added screen ids. For the sake of troubleshooting, I made a generated config and was able to add the fields. Here's the error now, it looks like it's screen assignment issues. Also to note, log levels being set to a numeric value in the generated config cause it it crash and give a key error on the numeric "10" or "20". Replacing with "debug" or "info" fixes this. Here's the config. First few lines are my print inserts.

{'filename': '/var/log/tenable.log', 'format': '%(asctime)-15s %(name)s %(levelname)s %(message)s', 'level': 'debug'} LOOK HERE: {'expand': 'description,lead,issueTypes,url,projectKeys,permissions,insight', 'self': 'https://REDACTED.atlassian.net/rest/api/3/project/12757', 'id': '12757', 'key': 'VULN', 'description': 'Managing vulnerabilities discovered from Tenable products.', 'lead': {'self': 'https://REDACTED.atlassian.net/rest/api/3/user?accountId=REDACTED', 'accountId': 'REDACTED', 'avatarUrls': {'48x48': 'https://secure.gravatar.com/avatar/93d838d1c56a0554bc9b732695d8aec4?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FST-5.png', '24x24': 'https://secure.gravatar.com/avatar/93d838d1c56a0554bc9b732695d8aec4?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FST-5.png', '16x16': 'https://secure.gravatar.com/avatar/93d838d1c56a0554bc9b732695d8aec4?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FST-5.png', '32x32': 'https://secure.gravatar.com/avatar/93d838d1c56a0554bc9b732695d8aec4?d=https%3A%2F%2Favatar-management--avatars.us-west-2.prod.public.atl-paas.net%2Finitials%2FST-5.png'}, 'displayName': 'svc tenable', 'active': True}, 'components': [], 'issueTypes': [{'self': 'https://REDACTED.atlassian.net/rest/api/3/issuetype/3', 'id': '3', 'description': 'A task that needs to be done.', 'iconUrl': 'https://REDACTED.atlassian.net/secure/viewavatar?size=medium&avatarId=11118&avatarType=issuetype', 'name': 'Task', 'subtask': False, 'avatarId': 11118}, {'self': 'https://REDACTED.atlassian.net/rest/api/3/issuetype/5', 'id': '5', 'description': 'The sub-task of the issue', 'iconUrl': 'https://REDACTED.atlassian.net/secure/viewavatar?size=medium&avatarId=11116&avatarType=issuetype', 'name': 'Sub-task', 'subtask': True, 'avatarId': 11116}], 'url': 'https://tenable.com', 'assigneeType': 'UNASSIGNED', 'versions': [], 'name': 'Vulnerability Management', 'roles': {'atlassian-addons-project-access': 'https://REDACTED.atlassian.net/rest/api/3/project/12757/role/10400', 'Service Desk Team': 'https://REDACTED.atlassian.net/rest/api/3/project/12757/role/10301', 'Developers': 'https://REDACTED.atlassian.net/rest/api/3/project/12757/role/10001', 'Service Desk Customers': 'https://REDACTED.atlassian.net/rest/api/3/project/12757/role/10300', 'Administrators': 'https://REDACTED.atlassian.net/rest/api/3/project/12757/role/10002', 'Users': 'https://REDACTED.atlassian.net/rest/api/3/project/12757/role/10000', 'Read Only': 'https://REDACTED.atlassian.net/rest/api/3/project/12757/role/10100', 'Tempo Project Managers': 'https://REDACTED.atlassian.net/rest/api/3/project/12757/role/10200'}, 'avatarUrls': {'48x48': 'https://REDACTED.atlassian.net/secure/projectavatar?pid=12757&avatarId=15423', '24x24': 'https://REDACTED.atlassian.net/secure/projectavatar?size=small&s=small&pid=12757&avatarId=15423', '16x16': 'https://REDACTED.atlassian.net/secure/projectavatar?size=xsmall&s=xsmall&pid=12757&avatarId=15423', '32x32': 'https://REDACTED.atlassian.net/secure/projectavatar?size=medium&s=medium&pid=12757&avatarId=15423'}, 'projectTypeKey': 'business', 'simplified': False, 'style': 'classic', 'isPrivate': False, 'properties': {}} LOOK HERE: {'errorMessages': [], 'errors': {'customfield_11601': "Field 'customfield_11601' cannot be set. It is not on the appropriate screen, or unknown.", 'customfield_11600': "Field 'customfield_11600' cannot be set. It is not on the appropriate screen, or unknown.", 'customfield_11603': "Field 'customfield_11603' cannot be set. It is not on the appropriate screen, or unknown.", 'customfield_11602': "Field 'customfield_11602' cannot be set. It is not on the appropriate screen, or unknown.", 'customfield_11595': "Field 'customfield_11595' cannot be set. It is not on the appropriate screen, or unknown.", 'customfield_11594': "Field 'customfield_11594' cannot be set. It is not on the appropriate screen, or unknown.", 'customfield_11597': "Field 'customfield_11597' cannot be set. It is not on the appropriate screen, or unknown.", 'customfield_11596': "Field 'customfield_11596' cannot be set. It is not on the appropriate screen, or unknown.", 'customfield_11599': "Field 'customfield_11599' cannot be set. It is not on the appropriate screen, or unknown.", 'customfield_11621': "Field 'customfield_11621' cannot be set. It is not on the appropriate screen, or unknown.", 'customfield_11598': "Field 'customfield_11598' cannot be set. It is not on the appropriate screen, or unknown."}}

Configuration File: closed_transitions:

Available IssueTypes 5: Sub-task 3: Task 10506: Task 10527: Subtask 10525: Task 10536: Task 10538: Subtask 10556: Task 10558: Subtask 10561: Task 10563: Subtask /-------------------------------NOTICE----------------------------------- | The output above is helpful for us to troubleshoot exactly what is | | happening within the code and offer a diagnosis for how to correct. | | Please note that while some basic redaction has already been performed | | that we ask you to review the information you're about to send and |

patrickoneiltc commented 3 years ago

The script is not accepting my screen definitions, as is shown below. It's still trying to use the screens that I'm specifically trying to avoid:

jira_id: '3'
name: Task
search:
Tenable Plugin ID
type: standard
jira_id: '5'
name: Sub-task
search:
Tenable Platform
Tenable Plugin ID
Tenable Asset UUID
Device IPv4 Addresses
Device IPv6 Addresses
Vulnerability Port
Vulnerability Protocol
type: subtask
jira:
address:
api_token:
api_username:
log:
filename: /var/log/tenable.log
format: '%(asctime)-15s %(name)s %(levelname)s %(message)s'
level: 10
project:
assigneeType: UNASSIGNED
description: Managing vulnerabilities discovered from Tenable products.
key: VULN
leadAccountId:
name: Vulnerability Management
projectTemplateKey: com.atlassian.jira-core-project-templates:jira-core-simplified-task-tracking
projectTypeKey: business
url: https://tenable.com
screen:
jira_ids:
12246
12247

Here's the settings from the config file. Did I set the values wrong?

291 project:
292   assigneeType: UNASSIGNED
293   description: Managing vulnerabilities discovered from Tenable products.
294   key: VULN
295   leadAccountId: [REDACTED]
296   name: Vulnerability Management
297   projectTemplateKey: com.atlassian.jira-core-project-templates:jira-core-simplified-task-tracking
298   projectTypeKey: business
299   url: https://tenable.com
300 screen:
301   jira_ids:
302   - 12246
303   - 12247
304   name:
305   - Task Management Edit/View Issue Screen
306   - Task Management Create Issue Screen
307   no_create: false
308   tabs:
patrickoneiltc commented 3 years ago

Can we continue troubleshooting over email? I'm still dead in the water over here.

patrickoneiltc commented 3 years ago

A code change to remove resftly accidentally deleted some auth for screen creation, breaking the process. Closing.