stix2.exceptions.InvalidValueError: Invalid value for Report 'created': must be a datetime object, date object, or timestamp string in a recognizable format.

[happyCodingGirl-XD]

We have been using version 1.2.1 and it worked perfectly fine for datetime with ISO format. Today, I upgraded to version 3.0.1, it can't parse ISO datetime anymore.

Test to reproduce the error:

from stix2 import Report
stix_report = Report(        
                                created     = '2021-09-26T06:32:30.035769',               
                                published   = '2021-10-26T06:00:30.012346',
                                modified    = '2021-09-29T06:01:35.015556',                    
                                name        = 'Test Report',
                                object_refs = 'identity--9293b79f-f935-45d3-9ff5-2f99154cce53',
                                object_marking_refs = 'marking-definition--3435bc16-43ba-46ab-9b9d-668ae141e768',
                                description = 'This is at test',
                                labels      = 'test-report'
                            )

This is the error:

 ValueError                                Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/stix2/utils.py in parse_into_datetime(value, precision, precision_constraint)
    248         try:
--> 249             parsed = dt.datetime.strptime(value, fmt)
    250         except (TypeError, ValueError):

7 frames
ValueError: time data '2021-09-26T06:32:30.035769' does not match format '%Y-%m-%dT%H:%M:%S.%fZ'

During handling of the above exception, another exception occurred:

ValueError                                Traceback (most recent call last)
ValueError: must be a datetime object, date object, or timestamp string in a recognizable format.

The above exception was the direct cause of the following exception:

InvalidValueError                         Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/stix2/base.py in _check_property(self, prop_name, prop, kwargs, allow_custom)
     58                 raise InvalidValueError(
     59                     self.__class__, prop_name, reason=str(exc),
---> 60                 ) from exc
     61 
     62         return has_custom

InvalidValueError: Invalid value for Report 'created': must be a datetime object, date object, or timestamp string in a recognizable format.

[chisholm]

Well, iso8601 defines various formats. STIX requires one particular ISO format. Your timestamps are missing "Z"s on the ends. Adding Z implies a zero UTC timezone offset, so if your timestamps are relative to a different timezone, you may need to adjust them before appending the Z.

Yeah, we did change datetime libraries awhile back, to address performance issues (see https://github.com/oasis-open/cti-python-stix2/pull/386 , https://github.com/oasis-open/cti-python-stix2/pull/397). We now use Python's builtin module. It performs better, but there was a tradeoff that there is less flexibility in accepted formats.

[happyCodingGirl-XD]

Thanks for the quick response. Yeah, I know it would work if I have the "Z" at the end. However, we use this in our automation to create the Report and other stix objects from the data we pull from ThreatStream. Their dates don't have the "Z" at the end. This breaks our automation procedure that worked before. We would have to write some custom code to add the "Z" to their dates or do something else. Do you have any suggestion? Thanks

[ramayer]

If anyone needs a workaround, you can put

import stix2.utils
class stix_old_date_style(object):
    def __enter__(self):
        self._TIMESTAMP_FORMAT      = stix2.utils._TIMESTAMP_FORMAT
        self._TIMESTAMP_FORMAT_FRAC = stix2.utils._TIMESTAMP_FORMAT_FRAC
        stix2.utils._TIMESTAMP_FORMAT = "%Y-%m-%dT%H:%M:%S"
        stix2.utils._TIMESTAMP_FORMAT_FRAC = "%Y-%m-%dT%H:%M:%S.%f"
        return self
    def __exit__(self ,type, value, traceback):
        stix2.utils._TIMESTAMP_FORMAT = self._TIMESTAMP_FORMAT
        stix2.utils._TIMESTAMP_FORMAT_FRAC = self._TIMESTAMP_FORMAT_FRAC
        return False

with stix_old_date_style() as s:
    stix_report = Report(....)

around your stix_report = Report( ...) block.

[chisholm]

I don't have a simple workaround. Ramayer's is ugly, but I guess it would work. Of course, those module-level variables would not be considered a public API. So there would be no promise of its stability.

[clenk]

Depending on how your automation works, just adding a Z to the end is probably simpler than ramayer's clever workaround. We could add code to check for special cases like this, but it would slow the library down and we're bound to miss other formatting variations.

[happyCodingGirl-XD]

Thanks, I ended up writing a function to add the Z to all the dates if there is not already a Z in them