Get an Error when

[dsouza713]

When used the integration script for tenable with JIRA cloud, I received the following error: 2020-02-21 11:23:12,528 restfly.errors.BadRequestError ERROR [400: POST] https://jirainstance.atlassian.net/rest/api/3/issue?update_history=False body={"errorMessages":[],"errors":{"customfield_10029":"data was not an array"}}

Note: that the fields were populated within JIRA, it's just that the data was not populated.

[dsouza713]

Did a debug and it looks like, it is going out of array: ERROR Message we receive: 2020-02-24 10:09:54,457 restfly.errors.BadRequestError ERROR [400: POST] https://jirainstance.atlassian.net/rest/api/3/issue?update_history=False body={"errorMessages":[],"errors":{"customfield_10029":"data was not an array"}}

Debug of the logs: "customfield_10029": "CVE-2019-8069,CVE-2019-8070",

it looks like it is going out of array because there are 2 CVE's and that's what is confusing it

[SteveMcGrath]

what does your configuration look like? The CVE configuration is a labels type, which should always be presented as an array.

[dsouza713]

I used the standard configuration:

tenable:
  platform: tenable.sc
  address: *********
  query_id: 22708
  username: *********
  password: *********

jira:
  api_token: *********
  api_username: *********
  address: jirainstance.atlassian.net

project:
  leadAccountId: *********

log:
  # What is the logging level desired?  Available levels are:
  #   #   debug, info, warn, error
  #     # The default level if unspecified is "warn"
  level: debug
  #
  #         # If you would like the log output to goto a file instead of standard output,
  #           # then specify the filename here:
  filename: /home/logfile.log

[SteveMcGrath]

what version is being used?

[dsouza713]

Version of?

[SteveMcGrath]

the integration.

[dsouza713]

Sure, It is: INFO Tenable2JiraCloud Version 1.1.1

Also, I tried another example where there was no CVE's for a vulnerability. It then failed whilst importing an IP: The error received was: restfly.errors.BadRequestError ERROR [400: POST] https://jirainstance.atlassian.net/rest/api/3/issue?update_history=False body={"errorMessages":[],"errors":{"customfield_10040":"data was not an array"}}

What was customfield_10040? "customfield_10040": "10.1.1.1"

Looks like it doesn't like separates values when it sees '.' ',' or '-'. Which is what makes it go out of the array. I am not sure as I am not an expert.

[SteveMcGrath]

the field -> custom field conversion is different for every Jira instance, as the script has to create those fields and them map them back to what we'd expect.

Version 1.1.1 will inform you of the mapping that has happened at the beginning of the log output, however I assume that customfield_10040 if the IPv4 field in your case. Whats odd is that the default config sets IPv4 as a label and labels should automatically be converted to arrays

https://github.com/tenable/integration-jira-cloud/blob/070f8710d5a701cc8b56286abb7193f43abce022/tenable_jira/transform.py#L235-L243

Can you set the log level to debug and paste the output of the last several lines before the error hits? I'd also need the beginning of the log as well that has the mappings.

[r1ckyb0nd]

Here is the debugging log for the 10040 error. 2020-02-24 10:43:06,813 urllib3.connectionpool DEBUG https://Companytest.atlassian.net:443 "POST /rest/api/3/search HTTP/1.1" 200 None 2020-02-24 10:43:06,816 tenable_jira.jira.Jira DEBUG uri=https://Companytest.atlassian.net/rest/api/3/issue, query={'update_history': False}, body={"fields": {"parent": {"key": "VULN-1"}, "customfield_10032": "10.0", "customfield_10030": "10.0", "customfield_10037": "Critical", "customfield_10036": "Wireshark / Ethereal Unsupported Version Detection", "customfield_10035": "Misc.", "customfield_10034": "56710", "customfield_10028": "Tenable.sc", "customfield_10053": "Test Vulns", "summary": "[10.4.2.196/0/TCP] [56710] Wireshark / Ethereal Unsupported Version Detection", "project": {"key": "VULN"}, "customfield_10043": "MSOHOSTING\\CHC-MABEL-D1W", "customfield_10040": "10.4.2.196", "customfield_10052": "145", "customfield_10046": "2020-01-19T23:53:25.000+0000", "customfield_10047": "2020-01-19T23:53:25.000+0000", "issuetype": {"id": "10003"}, "customfield_10050": "0", "customfield_10051": "TCP", "description": {"content": [{"content": [{"text": "Description", "type": "text"}], "type": "heading", "attrs": {"level": 1}}, {"content": [{"text": "According to its version, the installation of Wireshark / Ethereal on the remote host is no longer supported.\n\nLack of support implies that no new security patches for the product will be released by the vendor. As a result, it is likely to contain security vulnerabilities.", "type": "text"}], "type": "paragraph"}, {"content": [{"text": "Solution", "type": "text"}], "type": "heading", "attrs": {"level": 1}}, {"content": [{"text": "Upgrade to a version of Wireshark that is currently supported.", "type": "text"}], "type": "paragraph"}, {"content": [{"text": "Output", "type": "text"}], "type": "heading", "attrs": {"level": 1}}, {"content": [{"text": "No Output", "type": "text"}], "type": "paragraph"}], "version": 1, "type": "doc"}}} 2020-02-24 10:43:07,001 urllib3.connectionpool DEBUG https://Companytest.atlassian.net:443 "POST /rest/api/3/issue?update_history=False HTTP/1.1" 400 None 2020-02-24 10:43:07,003 restfly.errors.BadRequestError ERROR [400: POST] https://Companytest.atlassian.net/rest/api/3/issue?update_history=False body={"errorMessages":[],"errors":{"customfield_10040":"data was not an array"}}

[r1ckyb0nd]

And here is the debugging log for when it fails on 10029 CVEs

2020-02-24 10:31:47,411 urllib3.connectionpool DEBUG https://Companytest.atlassian.net:443 "POST /rest/api/3/search HTTP/1.1" 200 None 2020-02-24 10:31:47,413 tenable_jira.jira.Jira DEBUG uri=https://Companytest.atlassian.net/rest/api/3/issue, query={'update_history': False}, body={"fields": {"customfield_10033": "8.5", "customfield_10032": "9.8", "customfield_10031": "7.4", "customfield_10030": "10.0", "customfield_10037": "Critical", "customfield_10036": "Security Updates for Microsoft .NET Framework (September 2018)", "customfield_10035": "Windows : Microsoft Bulletins", "customfield_10034": "117431", "customfield_10029": "CVE-2018-8421", "summary": "[117431] Security Updates for Microsoft .NET Framework (September 2018)", "project": {"key": "VULN"}, "customfield_10054": "5.9", "issuetype": {"id": "10002"}, "description": {"content": [{"content": [{"text": "Description", "type": "text"}], "type": "heading", "attrs": {"level": 1}}, {"content": [{"text": "The Microsoft .NET Framework installation on the remote host is missing a security update. It is, therefore, affected by the following vulnerability :\n\n - A remote code execution vulnerability exists when Microsoft .NET Framework processes input. An attacker who successfully exploited this vulnerability could take control of an affected system. (CVE-2018-8421)", "type": "text"}], "type": "paragraph"}, {"content": [{"text": "Solution", "type": "text"}], "type": "heading", "attrs": {"level": 1}}, {"content": [{"text": "Microsoft has released security updates for Microsoft .NET Framework.", "type": "text"}], "type": "paragraph"}], "version": 1, "type": "doc"}}} 2020-02-24 10:31:47,569 urllib3.connectionpool DEBUG https://Companytest.atlassian.net:443 "POST /rest/api/3/issue?update_history=False HTTP/1.1" 400 None 2020-02-24 10:31:47,572 restfly.errors.BadRequestError ERROR [400: POST] https://Companytest.atlassian.net/rest/api/3/issue?update_history=False body={"errorMessages":[],"errors":{"customfield_10029":"data was not an array"}} 2020-02-24 10:35:00,377 root INFO Tenable2JiraCloud Version 1.1.1

[r1ckyb0nd]

In JIRA those fields are set to labels.

If the debugging log shows what is sent to JIRA, it looks like for customfield_10029 it is sending "customfield_10029": "CVE-2018-8421" when shouldn't it send "customfield_10029": ["CVE-2018-8421"] to make it an array?

[dsouza713]

Also, another thing I wanted to inform was that the script created the issue in Jira even tho it threw up the error for IP field i.e. customfield_10040, but the issue didn't have the IP obviously within Jira

However when we received the errors with the CVEs field ({"customfield_10029":"data was not an array"}), it didn't create any issues within Jira.

[dsouza713]

Hey Steve, I just tweeked the code, to pass the value as an array: if value:

for text-type fields, only sent the field if there is some

            # sort of data in it and recast the field as a string.
            if f['type'] in ['readonlyfield', 'textarea']:
                processed = str(value)

            # for labels, just pass on the field as-is
            elif f['type'] in ['labels']:
                if isinstance(value, str):
                    if fid == 'tsc_field':
                        processed = value.split(',')
                    else:
                        processed = [value,]
                else:
                    processed = [value]

and hooray, it worked i.e. the script didn't throw any errors and created an issue within Jira without any issues

[dsouza713]

Ran into another issue this morning, when there is too many CVE's associated with a vulnerbilty, it exceeds the character limit for labels in JIRA: restfly.errors.BadRequestError: [400: POST] https://jirainstance.atlassian.net/rest/api/3/issue?update_history=False body={"errorMessages":[],"errors":{"customfield_10029":"The label 'CVE-2019-8064,CVE-2019-8160,CVE-2019-8161,CVE-2019-8162,CVE-2019-8163,CVE-2019-8164,CVE-2019-8165,CVE-2019-8166,CVE-2019-8167,CVE-2019-8168,CVE-2019-8169,CVE-2019-8170,CVE-2019-8171,CVE-2019-8172,CVE-2019-8173,CVE-2019-8174,CVE-2019-8175,CVE-2019-8176,CVE-2019-8177,CVE-2019-8178,CVE-2019-8179,CVE-2019-8180,CVE-2019-8181,CVE-2019-8182,CVE-2019-8183,CVE-2019-8184,CVE-2019-8185,CVE-2019-8186,CVE-2019-8187,CVE-2019-8188,CVE-2019-8189,CVE-2019-8190,CVE-2019-8191,CVE-2019-8192,CVE-2019-8193,CVE-2019-8194,CVE-2019-8195,CVE-2019-8196,CVE-2019-8197,CVE-2019-8198,CVE-2019-8199,CVE-2019-8200,CVE-2019-8201,CVE-2019-8202,CVE-2019-8203,CVE-2019-8204,CVE-2019-8205,CVE-2019-8206,CVE-2019-8207,CVE-2019-8208,CVE-2019-8209,CVE-2019-8210,CVE-2019-8211,CVE-2019-8212,CVE-2019-8213,CVE-2019-8214,CVE-2019-8215,CVE-2019-8216,CVE-2019-8217,CVE-2019-8218,CVE-2019-8219,CVE-2019-8220,CVE-2019-8221,CVE-2019-8222,CVE-2019-8223,CVE-2019-8224,CVE-2019-8225,CVE-2019-8226' exceeds the maximum length for a single label of 255 characters."}}

I guess a solution would be to exclude the remaining CVE's and continue with the import. Because currently when this error throws up, it stops the import

[dsouza713]

So after some debugging and looking at logs, it looks like the CVEs were being passed as one string i.e. all cramped into one value. which was causing it to exceed the 255 character limit.

So i changed the below code for it to split the CVE's into individual values and then pass it in an array the way Jira likes it:

for labels, just pass on the field as-is

            elif f['type'] in ['labels']:
                if isinstance(value, str):
                    if fid == 'tsc_field':
                        processed = value.split(',')
                    else:
                        processed = [value,]
                else:
                    processed = [ x.strip() for x in value.strip('[]').split(',') ]

A question that we had was why it does not get picked up by the tsc_field if statement. Because by the looks of it, it gets processed by the last 'else' statement

[SteveMcGrath]

I was trying to figure that out, as I'm not experiencing that myself. As you're already mucking about in the code, why dont we revert your changes and instead look to see what the code is seeing. Add the following before L237:

                    self._log.debug('Label Detected.  Config={} value={}'.format(str(f), str(value)))

If should be right after the elif f['type'] in ['labels']: statement. In this case we want this to fail, we just need more info.

[SteveMcGrath]

Are you using python 2 or 3?

[r1ckyb0nd]

what version should we be using?

[SteveMcGrath]

3.6+

[r1ckyb0nd]

This can be closed after forcing the version of python to 3.6.8

[SteveMcGrath]

excellent. readme also updated to state the py3.6 requirement. it was listed as py3 only on pypi, but i guess that wont stop installing from happening.