Handle exceptions where no EPSS data exists

[himynamesdave]

Some CVE's have no EPSS data, e.g.

https://api.first.org/data/v1/epss?cve=CVE-2024-37031

Causes hard error (shown below). In these case, create the note object, but do not add any EPSS scores to it. We should create the object so it can be handled by downstream software (namely ACTIP) without issue,

2024-10-08 16:20:28,328 | INFO | CVE-> CVE-2024-37031
2024-10-08 16:20:28,854 | ERROR | Chord '5fa0c065-0b5c-4d8c-b833-8e3cb3865bf0' raised: ChordError("Dependency 116f07fb-a547-49ae-873b-60a6e8dfae32 raised KeyError('date')")
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/celery/app/trace.py", line 477, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/celery/app/trace.py", line 760, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dgreenwood/Documents/repos/dogesec/cve2stix/cve2stix/celery.py", line 35, in cve_syncing_task
    fetch_data(start, end, config)
  File "/Users/dgreenwood/Documents/repos/dogesec/cve2stix/cve2stix/main.py", line 28, in fetch_data
    return fetch_url(urlunsplit(uri), config, parse_cve_api_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dgreenwood/Documents/repos/dogesec/cve2stix/cve2stix/utils.py", line 49, in fetch_url
    response = callback(content, config)
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dgreenwood/Documents/repos/dogesec/cve2stix/cve2stix/parse_api_response.py", line 272, in parse_cve_api_response
    config.fs.add(parse_cve_epss_note(cve, vulnerability, config))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dgreenwood/Documents/repos/dogesec/cve2stix/cve2stix/parse_api_response.py", line 223, in parse_cve_epss_note
    modified=datetime.strptime(epss_data["date"], "%Y-%m-%d").date(),
                               ~~~~~~~~~^^^^^^^^
KeyError: 'date'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/celery/backends/redis.py", line 528, in on_chord_part_return
    resl = [unpack(tup, decode) for tup in resl]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/celery/backends/redis.py", line 528, in <listcomp>
    resl = [unpack(tup, decode) for tup in resl]
            ^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/celery/backends/redis.py", line 434, in _unpack_chord_result
    raise ChordError(f'Dependency {tid} raised {retval!r}')
celery.exceptions.ChordError: Dependency 116f07fb-a547-49ae-873b-60a6e8dfae32 raised KeyError('date')
2024-10-08 16:20:28,862 | ERROR | Task cve2stix.celery.cve_syncing_task[116f07fb-a547-49ae-873b-60a6e8dfae32] raised unexpected: KeyError('date')
Traceback (most recent call last):
  File "/opt/homebrew/lib/python3.11/site-packages/celery/app/trace.py", line 477, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/lib/python3.11/site-packages/celery/app/trace.py", line 760, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dgreenwood/Documents/repos/dogesec/cve2stix/cve2stix/celery.py", line 35, in cve_syncing_task
    fetch_data(start, end, config)
  File "/Users/dgreenwood/Documents/repos/dogesec/cve2stix/cve2stix/main.py", line 28, in fetch_data
    return fetch_url(urlunsplit(uri), config, parse_cve_api_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dgreenwood/Documents/repos/dogesec/cve2stix/cve2stix/utils.py", line 49, in fetch_url
    response = callback(content, config)
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dgreenwood/Documents/repos/dogesec/cve2stix/cve2stix/parse_api_response.py", line 272, in parse_cve_api_response
    config.fs.add(parse_cve_epss_note(cve, vulnerability, config))
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/dgreenwood/Documents/repos/dogesec/cve2stix/cve2stix/parse_api_response.py", line 223, in parse_cve_epss_note
    modified=datetime.strptime(epss_data["date"], "%Y-%m-%d").date(),
                               ~~~~~~~~~^^^^^^^^
KeyError: 'date

[himynamesdave]

@fqrious did you push this?

[himynamesdave]

@fqrious

We should create the object so it can be handled by downstream software (namely ACTIP) without issue,

The exception is now avoided. But no note object is created.

UPDATE: I realised this might happen b/c of modified / created time taking EPSS time. If not EPSS, the note should take the same created/modified time as Vulnerability.

IF EPSS exists, then use CVE time

[fqrious]

if there is no epss, it gets skipped... what's wrong with that?

[fqrious]

We should create the object so it can be handled by downstream software (namely ACTIP) without issue,

We don't need to create an object for ACTIP to work

[himynamesdave]

We should create the object so it can be handled by downstream software (namely ACTIP) without issue,

We don't need to create an object for ACTIP to work

OK, so just to confirm, if no Note object exists for a CVE. When ACTIP goes to update it (when a EPSS is published), ACTIP has the logic to create a brand new note to represent EPSS?

[fqrious]

Yes

On Wed, Oct 9, 2024, 6:42 AM David Greenwood @.***> wrote:

We should create the object so it can be handled by downstream software (namely ACTIP) without issue,

We don't need to create an object for ACTIP to work

OK, so just to confirm, if no Note object exists for a CVE. When ACTIP goes to update it (when a EPSS is published), ACTIP has the logic to create a brand new note to represent EPSS?

— Reply to this email directly, view it on GitHub https://github.com/muchdogesec/cve2stix/issues/26#issuecomment-2401362879, or unsubscribe https://github.com/notifications/unsubscribe-auth/AHB3QISMHILARLE3YGKEDZTZ2S625AVCNFSM6AAAAABPSQW6I6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMBRGM3DEOBXHE . You are receiving this because you were mentioned.Message ID: @.***>