tenable / integration-jira-cloud

69 stars 51 forks source link

Tenable.io to Jira integration doesn't import First Seen and Last Seen field data #45

Closed dil-ddsouza closed 4 years ago

dil-ddsouza commented 4 years ago

Hi Steve,

I noticed today that the integration doesn't populate the Vulnerability First Seen and Last seen data within Jira, although that data is present in tenable.

Here is an example: You can see the data is present in Tenable.io for that particular vulnerability: image

However, in Jira, the data doesn't get populated by the integration: image

Do you know why this would be the case please? Any help/advise would be appreciated.

P.S. we didn't notice such an issue with Tenable.sc to Jira.

SteveMcGrath commented 4 years ago

Those fields should be populated. Can you run an instance with --troubleshoot and report back with the output?

dil-ddsouza commented 4 years ago

Hi Steve,

Here is a snippet of the output: 2020-05-15 11:57:59,825 urllib3.connectionpool DEBUG https://:443 "POST /rest/api/3/search HTTP/1.1" 200 None 2020-05-15 11:57:59,826 tenable_jira.jira.Jira INFO UPDATED AGENTVULN-646 [ip-10-1-1-1/0/TCP] [105106] Ubuntu 14.04 LTS : linux vulnerabilities (USN-3510-1) (Dirty COW) 2020-05-15 11:57:59,827 tenable_jira.jira.Jira DEBUG {"method": "PUT", "url": "https:///rest/api/3/issue/386747", "params": {"notifyUsers": "true", "overrideScreenSecurity": "false", "overrideEditableFlag": "false"}, "body": {"fields": {"project": {"key": "AGENTVULN"}, "issuetype": {"id": 19}, "customfield_16097": "Tenable.io", "customfield_16099": "7.2", "customfield_16100": "2017-12-07T00:00:00.000+0000", "customfield_16101": "5.6", "customfield_16102": "7.8", "customfield_16103": " 7.0", "customfield_16104": "105106", "customfield_16105": "Ubuntu Local Security Checks", "customfield_16106": "Ubuntu 14.04 LTS : linux vulnerabilities (USN-3510-1) (Dirty COW)", "customfield_16107": "High", "customfield_16108": ["24f395b2-73b9-4d90-b9 0c-cea0e50f114b"], "customfield_16109": ["0A:59:26:5E:70:5D"], "customfield_16110": ["10.1.1.1"], "customfield_16112": ["ip-10-1-1-1"], "customfield_16114": "ip-10-1-1-1", "customfield_16115": "00000000-0000-0000-0000-000000000000", "customf ield_16119": "OPEN", "customfield_16121": "TCP", "customfield_16124": "6.7", "summary": "[ip-10-1-1-1/0/TCP] [105106] Ubuntu 14.04 LTS : linux vulnerabilities (USN-3510-1) (Dirty COW)", "description": {"version": 1, "type": "doc", "content": [{"type ": "heading", "attrs": {"level": 1}, "content": [{"type": "text", "text": "Description"}]}, {"type": "paragraph", "content": [{"type": "text", "text": "Mohamed Ghannam discovered that a use-after-free vulnerability existed\nin the Netlink subsystem (XFR M) in the Linux kernel. A local attacker\ncould use this to cause a denial of service (system crash) or possibly\nexecute arbitrary code. (CVE-2017-16939)\n\nIt was discovered that the Linux kernel did not properly handle\ncopy-on- write of transparent huge pages. A local attacker could use\nthis to cause a denial of service (application crashes) or possibly\ngain administrative privileges. (CVE-2017-1000405).\n\nNote that Tenable Network Security has extracted the preceding\ndescription block directl y from the Ubuntu security advisory. Tenable\nhas attempted to automatically clean and format it as much as possible\nwithout introducing additional issues."}]}, {"type": "heading", "attrs": {"level": 1}, "content": [{"type": "text", "text": "Solution"} ]}, {"type": "paragraph", "content": [{"type": "text", "text": "Update the affected packages."}]}, {"type": "heading", "attrs": {"level": 1}, "content": [{"type": "text", "text": "Output"}]}, {"type": "paragraph", "content": [{"type": "text", "text": "\ n - Installed package : linux-image-3.13.0-74-generic_3.13.0-74.118\n Fixed package : linux-image-3.13.0-137-generic_3.13.0-137.186\n\n"}]}]}, "parent": {"key": "AGENTVULN-645"}}}}

As you can see from the above snippet, the following custom fields are missing. Which are the first and last seen fields: customfield_16116 - Vulnerability First Seen customfield_16117 - Vulnerability Last Seen

It is is the same for all the vulns.

SteveMcGrath commented 4 years ago

That doesn't make any sense... Can you provide your config?

dil-ddsouza commented 4 years ago
import yaml

def base_config():
    return yaml.load(config, Loader=yaml.Loader)

# WARNING: These are the default values that control how the transformer
#          processes vulnerability data into Jira tickets.  While the code
#          itself is meant to be very flexible, it's quite easy to shoot
#          yourself in the foot.  The complete configuration has therefor
#          been vendored below and parameters are overridden from the supplied
#          config file.  Overloading any values not explicitly documented in
#          the documentation is considered custom modification and will not be
#          officially supported as part of this out-of-the box integration.
#
#          In short, MODIFY AT YOUR OWN RISK.

config = '''
tenable:
  # What platform should we be connecting to?
  #  Must be either tenable.io and tenable.sc
  platform: tenable.io

  # Tenable.io or Tenable.sc API Access Key
  access_key:

  # Tenable.io or Tenable.sc API Secret Key
  secret_key:

  # The hostname for the Tenable.sc host
  address:

  # The port number on Tenable.sc to connect to
  port: 443

  # Note that Tenable.sc supports either session authentication or key
  # authentication.  You only need to provide one or the other.
  #
  # The username to use for Tenable.sc session auth.
  username:

  # The password to use for Tenable.sc session auth.
  password:

  # Tenable.io vulnerability severities to convert to JIRA tickets.
  tio_severities:
    - high
    - critical

  # Tenable.sc Query to use as the basis for generating JIRA tickets.
  query_id:

  # Number of assets per chunk to export from Tenable.io.
  chunk_size: 1000

  # Page size for Tenable.sc Analysis calls.
  page_size: 1000

jira:
  # The API Token to use to authenticate to the Jira application
  api_token:

  # The User that will be authenticaing to the Jira application
  api_username:

  # The address pointing to the Jira application.
  address: your-domain.atlassian.net

# The project definition is passed directly to the project creator if no project
# by the specified key does not exist.
# https://developer.atlassian.com/cloud/jira/platform/rest/v3/?utm_source=%2Fcloud%2Fjira%2Fplatform%2Frest%2F&utm_medium=302#api-rest-api-3-project-get
project:
  # The project Key to use.
  key: AGENTVULN

  # The name of the project
  name: Agent Vulnerability Management

  # The project type
  projectTypeKey: business

  # The Jira project template id.
  projectTemplateKey: com.atlassian.jira-core-project-templates:jira-core-simplified-task-tracking

  # A description for the project
  description: Managing vulnerabilities discovered from Tenable products.

  # URL for the project.,
  url: https://tenable.com

  # The assignee determination for new issues.  Must be either
  # UNASSIGNED or PROJECT_LEAD as per the API docs.
  assigneeType: UNASSIGNED

  # The UUID for the project lead user.
  leadAccountId:

# This section defines the issue-types & how to search them.  There should only
# ever be a singular "standard" and no more than 1 "subtask".  The issue-type
# name is what will be used throughout the rest of the config file to determine
# what issue-type gets what data fields.
issue_types:
  - jira_id: 18
    name: Task
    type: standard
    search:
      - Tenable Plugin ID
  - jira_id: 19
    name: Sub-task
    type: subtask
    search:
      - Tenable Platform
      - Tenable Plugin ID
      - Tenable Asset UUID
      - Device IPv4 Addresses
      - Device IPv6 Addresses
      - Vulnerability Port
      - Vulnerability Protocol

# What transitions should be considered closed?
closed_transitions:
  - Closed
  - Done
  - Resolved

# Jira issues have some predefined fields.  When leveraging those, we will want
# to define how to use them here.  As the same field can be used differently
# depending on issue-type, we define how to use the field per issue-type.  When
# defining a vulnerability field or fields to use, we will use a parameterized
# string in the python standard.  All vuln items are passed as the vuln dict and
# the keys reference the flattened dictionary structure.
#
# Simple Single-line Example:
# summary:        <-- The Jira Field Name
#   Task:         <-- The Issue Type
#     tio_field:  <-- What to do with Tenable.io data
#     tsc_field:  <-- What to do with Tenable.sc data
#
# Multi-Paragraph Example:
# description:    <-- The Jira Field Name
#   Task:         <-- The Issue Type
#     - name:     <-- The name of the document section
#       tio_field <-- What to do with Tenable.io data
#       tsc_field <-- What to do with Tenable.sc data
issue_default_fields:
  summary:
    Task:
      tio_field: '[{vuln[plugin.id]}] {vuln[plugin.name]}'
      tsc_field: '[{vuln[pluginID]}] {vuln[pluginName]}'
    Sub-task:
      tio_field: '[{vuln[asset.hostname]}/{vuln[port.port]}/{vuln[port.protocol]}] [{vuln[plugin.id]}] {vuln[plugin.name]}'
      tsc_field: '[{vuln[ip]}/{vuln[port]}/{vuln[protocol]}] [{vuln[pluginID]}] {vuln[pluginName]}'
  description:
    Task:
      - name: Description
        tio_field: '{vuln[plugin.description]}'
        tsc_field: '{vuln[description]}'
      - name: Solution
        tio_field: '{vuln[plugin.solution]}'
        tsc_field: '{vuln[solution]}'
    Sub-task:
      - name: Description
        tio_field: '{vuln[plugin.description]}'
        tsc_field: '{vuln[description]}'
      - name: Solution
        tio_field: '{vuln[plugin.solution]}'
        tsc_field: '{vuln[solution]}'
      - name: Output
        tio_field: '{vuln[output]}'
        tsc_field: '{vuln[pluginOutput]}'

# Screen definition section
screen:
  #jira_ids:
  #  - 14632
  #  - 14633
  # What screens should we be managing?
  name:
    - Task Management Edit/View Issue Screen
    - Task Management Create Issue Screen

  # How should the fields be laid out?  The tab name of "default" would specify
  # the main tab that you'd see when opening the issue.  Any other tabs are
  # sub-tabs that must be clicked into.
  tabs:
    Vulnerability:
      - CVEs
      - Tenable VPR Score
      - CVSSv2 Base Score
      - Patch Publish Date
      - CVSSv2 Temporal Score
      - CVSSv3 Base Score
      - CVSSv3 Temporal Score
      - Tenable Plugin ID
      - Tenable Plugin Family
      - Tenable Plugin Name
      - Vulnerability Severity
      - Vulnerability First Seen
      - Vulnerability Last Seen
      - Vulnerability Last Fixed
      - Vulnerability State
      - Vulnerability Port
      - Vulnerability Protocol
    Asset:
      - Tenable Asset UUID
      - Tenable Platform
      - Device Hostname
      - Device NetBIOS Name
      - Device DNS Name
      - Device IPv4 Addresses
      - Device IPv6 Addresses
      - Device MAC Addresses
      - Device Network ID
      - Vulnerability Repository ID
      - Vulnerability Repository Name

# The custom fields are created automatically if they do not exist.  Further the
# mapping between the jira_field and the tio_field & tsc_field indicate what
# data is passed into the Jira custom field.
fields:

# An example of a specified field is as follows:
# - jira_field: NAME    - Name of the Jira Field. We search for this w/in the API.
#   jira_id: ID         - If specified, use this field ID instead of creating.
#   type: DATATYPE      - The type of data that will be stored here.
#   searcher: SEARCHER  - The Jira searcher to use for searching in Jira
#   issue_type:         - The issue types that this field is to be associated with.
#     - TYPE1
#     - TYPE2
#   tio_field: field.name - Tenable.io field to parse for this JIRA field.
#   tsc_field: field.name - Tenable.sc field to parse for this JIRA field.

  - jira_field: Tenable Platform
    type: readonlyfield
    searcher: textsearcher
    is_platform_id: true
    issue_type:
      - Sub-Task

  # Vulnerability fields
  - jira_field: CVEs
    type: labels
    searcher: labelsearcher
    issue_type:
      - Task
      - Sub-Task
    tio_field: plugin.cve
    tsc_field: cve

  - jira_field: CVSSv2 Base Score
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Task
      - Sub-task
    tio_field: plugin.cvss_base_score
    tsc_field: baseScore

  - jira_field: Patch Publish Date
    type: datetime
    searcher: datetimerange
    issue_type:
      - Task
      - Sub-task
    tio_field: plugin.patch_publication_date
    tsc_field: patchPubDate

  - jira_field: CVSSv2 Temporal Score
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Task
      - Sub-task
    tio_field: plugin.cvss_temporal_score
    tsc_field: temporalScore

  - jira_field: CVSSv3 Base Score
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Task
      - Sub-task
    tio_field: plugin.cvss3_base_score
    tsc_field: cvssV3BaseScore

  - jira_field: CVSSv3 Temporal Score
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Task
      - Sub-task
    tio_field: plugin.cvss3_temporal_score
    tsc_field: cvssV3TemporalScore

  - jira_field: Tenable Plugin ID
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Task
      - Sub-task
    tio_field: plugin.id
    tsc_field: pluginID

  - jira_field: Tenable Plugin Family
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Task
      - Sub-task
    tio_field: plugin.family
    tsc_field: family.name

  - jira_field: Tenable Plugin Name
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Task
      - Sub-task
    tio_field: plugin.name
    tsc_field: pluginName

  - jira_field: Vulnerability Severity
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Task
      - Sub-task
    tio_field: plugin.risk_factor
    tsc_field: severity.name

  # Vulnerability Instance fields
  - jira_field: Tenable Asset UUID
    type: labels
    searcher: labelsearcher
    issue_type:
      - Sub-task
    tio_field: asset.uuid

  - jira_field: Device MAC Addresses
    type: labels
    searcher: labelsearcher
    issue_type:
      - Sub-task
    tio_field: asset.mac_address
    tsc_field: macAddress

  - jira_field: Device IPv4 Addresses
    type: labels
    searcher: labelsearcher
    issue_type:
      - Sub-task
    tio_field: asset.ipv4
    tsc_field: ip

  - jira_field: Device IPv6 Addresses
    type: labels
    searcher: labelsearcher
    issue_type:
      - Sub-task
    tio_field: asset.ipv6

  - jira_field: Device Hostname
    type: labels
    searcher: labelsearcher
    issue_type:
      - Sub-task
    tio_field: asset.hostname
    tsc_field: dnsName

  - jira_field: Device NetBIOS Name
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Sub-task
    tsc_field: netbiosName

  - jira_field: Device DNS Name
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Sub-task
    tio_field: asset.fqdn
    tsc_field: dnsName

  - jira_field: Device Network ID
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Sub-task
    tio_field: asset.network_id

  - jira_field: Vulnerability First Seen
    type: datetime
    searcher: datetimerange
    issue_type:
      - Sub-task
    tio_field: first_seen
    tsc_field: firstSeen

  - jira_field: Vulnerability Last Seen
    type: datetime
    searcher: datetimerange
    issue_type:
      - Sub-task
    tio_field: last_seen
    tsc_field: lastSeen

  - jira_field: Vulnerability Last Fixed
    type: datetime
    searcher: datetimerange
    issue_type:
      - Sub-task
    tio_field: last_fixed

  - jira_field: Vulnerability State
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Sub-task
    tio_field: state

  - jira_field: Vulnerability Port
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Sub-task
    tio_field: port.port
    tsc_field: port

  - jira_field: Vulnerability Protocol
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Sub-task
    tio_field: port.protocol
    tsc_field: protocol

  - jira_field: Vulnerability Repository ID
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Sub-task
    tsc_field: repository.id

  - jira_field: Vulnerability Repository Name
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Sub-task
    tsc_field: repository.name

  - jira_field: Tenable VPR Score
    type: readonlyfield
    searcher: textsearcher
    issue_type:
      - Task
      - Sub-task
    tio_field: plugin.vpr.score
    tsc_field: vprScore
dil-ddsouza commented 4 years ago

There you go Steve, hopefully you can spot something.

SteveMcGrath commented 4 years ago

have you modified the config.py file directly?

dil-ddsouza commented 4 years ago

Hi Steve,

The only changes I made was to the following:

Added the key and name for the project key: AGENTVULN name: Agent Vulnerability Management

Added id's as the script was not recognising the screens issue_types:

Added the following to import patch publish date: **- jira_field: Patch Publish Date type: datetime searcher: datetimerange issue_type:

SteveMcGrath commented 4 years ago

Alright, so I would highly recommend that you dont modify that file directly, as it'll block you from updates. Instead if you want to modify, then copy the contents to your config file and adjust there. Your config file will override the parameters in the vendored config.

dil-ddsouza commented 4 years ago

So you mean I can add the above stuff in the Config.yaml file? And the script will pick it up?

So something like this, added my stuff to the bottom:

tenable:
  access_key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
  secret_key: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx

jira:
  api_token: xxxxxxxxxxxxxxxxxxxx
  api_username: xxxxxxxxxxxxxxxxx
  address: jirainstance.net

project:
  leadAccountId: xxxxxxxxxxxxxxxxxxxxxx

log:
  # What is the logging level desired?  Available levels are:
  #   #   #   debug, info, warn, error
  #     #     # The default level if unspecified is "warn"
  level: debug
  #         #
  #           #         # If you would like the log output to goto a file instead of standard output,
  #             #           # then specify the filename here:
  filename: /home/ddsouza/Scripts/logfiletest.log

  # The following section is optional.  You should only specify this section if
  #   # you would like the bridge to run as a service with it's own timer.
#service:
  #         # How many hours should we wait between jobs?  Note setting this to a
  #             # non-zero number will result in imports past the initial import will use
  #                 # time of the last completed import as the basis for last observed.
 # interval: 3

key: AGENTVULN
name: Agent Vulnerability Management
jira_id: 18
jira_id: 19
- jira_field: Patch Publish Date
type: datetime
searcher: datetimerange
issue_type:
- Task
- Sub-task
tio_field: plugin.patch_publication_date
tsc_field: patchPubDate
SteveMcGrath commented 4 years ago

Things would need to be added to appropriate section. Also its an override, so in the case of the lists (such as the field declarations), you would need to pass all of them. I did push the addition of a "Patch Publication Date" field however, so if you rename you're custom field to "Patch Publication Date" it should simply use it. This would mean that your config.yaml would look like this:

tenable:
  access_key: xxxxxxxxxxxx
  secret_key: xxxxxxxxxxxx

jira:
  api_token: xxxxxxxxx
  api_username: xxxxxxxxx
  address: jirainstance

project:
  key: AGENTVULN
  leadAccountId: xxxxxxxx

log:
  level: debug
  filename: /home/ddsouza/Scripts/logfiletest.log
dil-ddsouza commented 4 years ago

Thanks for this Steve, I'll give this a go