attack_client().get_techniques() leading to Invalid JSON error

[Spider2themax]

Running the following:

from attackcti import attack_client
techniques = attack_client().get_techniques()

but getting this error... full trace here...

---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/requests/models.py:910, in Response.json(self, **kwargs)
    909 try:
--> 910     return complexjson.loads(self.text, **kwargs)
    911 except JSONDecodeError as e:
    912     # Catch JSON-related errors and raise as requests.JSONDecodeError
    913     # This aliases json.JSONDecodeError and simplejson.JSONDecodeError

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/simplejson/__init__.py:525, in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, use_decimal, **kw)
    521 if (cls is None and encoding is None and object_hook is None and
    522         parse_int is None and parse_float is None and
    523         parse_constant is None and object_pairs_hook is None
    524         and not use_decimal and not kw):
--> 525     return _default_decoder.decode(s)
    526 if cls is None:

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/simplejson/decoder.py:370, in JSONDecoder.decode(self, s, _w, _PY3)
    369     s = str(s, self.encoding)
--> 370 obj, end = self.raw_decode(s)
    371 end = _w(s, end).end()

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/simplejson/decoder.py:400, in JSONDecoder.raw_decode(self, s, idx, _w, _PY3)
    399         idx += 3
--> 400 return self.scan_once(s, idx=_w(s, idx).end())

JSONDecodeError: Unterminated string starting at: line 1 column 64468 (char 64467)

During handling of the above exception, another exception occurred:

JSONDecodeError                           Traceback (most recent call last)
File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/taxii2client/common.py:124, in _to_json(resp)
    123 try:
--> 124     return resp.json()
    125 except ValueError as e:
    126     # Maybe better to report the original request URL?

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/requests/models.py:917, in Response.json(self, **kwargs)
    916 else:
--> 917     raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)

JSONDecodeError: [Errno Unterminated string starting at] {"type":"bundle","id":"bundle--5c1e0454-6314-4d5b-af6d-8a50e518786a","spec_version":"2.0","objects":[{"object_marking_refs":["marking-definition--fa42a846-8d90-4e51-bc29-71d5b4802168"],"type":"attack-pattern","name":"Resource Forking","modified":"2021-10-16T01:50:40.276Z","created":"2021-10-12T20:02:31.866Z","kill_chain_phases":[{"kill_chain_name":"mitre-attack","phase_name":"defense-evasion"}]

leaving out part of string here...

...they may collect sensitive information such as proprietary source code or credentials contained within software's source code.  Having access to software's source code may allow adversaries to develop [Exploits](https://attack.mitre.: 64467

The above exception was the direct cause of the following exception:

InvalidJSONError                          Traceback (most recent call last)
Input In [3], in <module>
----> 1 techniques = attack_client().get_techniques()

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/attackcti/attack_api.py:932, in attack_client.get_techniques(self, include_subtechniques, skip_revoked_deprecated, enrich_data_sources, stix_format)
    919 """ Extracts all the available techniques STIX objects across all ATT&CK matrices
    920 
    921 Args: 
   (...)
    928     List of STIX objects
    929 """
    931 if include_subtechniques:
--> 932     all_techniques = self.COMPOSITE_DS.query(Filter("type", "=", "attack-pattern"))
    933 else:
    934     all_techniques = self.COMPOSITE_DS.query([
    935         Filter("type", "=", "attack-pattern"),
    936         Filter('x_mitre_is_subtechnique', '=', False)
    937     ])

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/stix2/datastore/__init__.py:570, in CompositeDataSource.query(self, query, _composite_filters)
    567 # federate query to all attached data sources,
    568 # pass composite filters to id
    569 for ds in self.data_sources:
--> 570     data = ds.query(query=query, _composite_filters=all_filters)
    571     all_data.extend(data)
    573 # remove exact duplicates (where duplicates are STIX 2.0
    574 # objects with the same 'id' and 'modified' values)

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/stix2/datastore/taxii.py:301, in TAXIICollectionSource.query(self, query, version, _composite_filters)
    299 paged_request = tcv21.as_pages if isinstance(self.collection, tcv21.Collection) else tcv20.as_pages
    300 try:
--> 301     for resource in paged_request(self.collection.get_objects, per_request=self.items_per_page, **taxii_filters_dict):
    302         all_data.extend(resource.get("objects", []))
    303 except HTTPError as e:
    304     # if resources not found or access is denied from TAXII server, return empty list

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/taxii2client/v20/__init__.py:36, in as_pages(func, start, per_request, *args, **kwargs)
     25 """Creates a generator for TAXII 2.0 endpoints that support pagination.
     26 
     27 Args:
   (...)
     33 Use args or kwargs to pass filter information or other arguments required to make the call.
     34 """
     35 resp = func(start=start, per_request=per_request, *args, **kwargs)
---> 36 yield _to_json(resp)
     37 total_obtained, total_available = _grab_total_items(resp)
     39 if total_available > per_request and total_obtained != per_request and total_obtained != float("inf"):

File ~/Library/Caches/pypoetry/virtualenvs/XXX-qNGBkjnK-py3.8/lib/python3.8/site-packages/taxii2client/common.py:127, in _to_json(resp)
    124     return resp.json()
    125 except ValueError as e:
    126     # Maybe better to report the original request URL?
--> 127     six.raise_from(InvalidJSONError(
    128         "Invalid JSON was received from " + resp.request.url
    129     ), e)

File <string>:3, in raise_from(value, from_value)

InvalidJSONError: Invalid JSON was received from https://cti-taxii.mitre.org/stix/collections/95ecc380-afe9-11e4-9b6c-751b66dd541e/objects/?match%5Btype%5D=attack-pattern

Thanks!

[Cyb3rWard0g]

Hello @Spider2themax !

I mentioned the following in the other issue you referenced in here. Just posting the same response to your comment. I believe it might have been an issue with the TAXII server and not the library itself. Would you mind testing it again?

[Cyb3rWard0g]

Hello @Spider2themax ! Let me know if this works. I checked with Luke whom opened a similar issue and it seems that everything is working fine now. Looked like it was a server-side issue.

[Spider2themax]

@Cyb3rWard0g yep it looks like that was definitely the issue. If there was some error catching to know if something is broken, that was the TAXII service being broken, that would be great just so we would know it wasnt from attackcti

[Spider2themax]

Resolved; not an issue with attackcti but a server-side issue