import stixv2.1 to MISP via pymisp

[tahaconfiant]

hello,

it seems that this function pymisp.tools.stix.load_stix() can only load stix v1. tried with stix v2, stix v2.1

the returned package doesnt contains objects or attributes, therefore when uploaded to MISP appears to be an empty event.

question is simple: I want to load stix v2.1 files with pymisp but doesnt seems to be working

see attached file for a sample stix v2.1 file

blast.json.zip

[chrisinmtown]

For what it's worth, you can import STIX2 data using the web UI. But you must install the stix2 Python package on the server. Did you make sure that's installed? I would also test the import on the server to make absolutely certain it works there.

[tahaconfiant]

the manual import using the web UI is working,I can import stix 2.0 and are perfectly parsed.

The issue relies on when I call PyMISP to parse a STIX v2.1 feed,

my code is the following

import pymisp
from io import StringIO

with open("blast.json", "rb") as r:
    block = r.read()

if isinstance(block, bytes):
    block = block.decode()
package = pymisp.tools.stix.load_stix(StringIO(block))
print (package)
values = [x.value for x in package.attributes]
print("Extracted %s", values)
print (package.objects)

I get the following errors:

loading STIX...
Traceback (most recent call last):
  File "/samples/load-stix.py", line 14, in <module>
    package = pymisp.tools.stix.load_stix(StringIO(block))
  File "/usr/local/lib/python3.9/site-packages/pymisp/tools/stix.py", line 17, in load_stix
    return buildEvent(stix, distribution=distribution,
  File "/Users/xxxx/Library/Python/3.9/lib/python/site-packages/misp_stix_converter/converters/buildMISPAttribute.py", line 480, in buildEvent
    event = parseIndicators(event, pkg)
  File "/Users/test/Library/Python/3.9/lib/python/site-packages/misp_stix_converter/converters/buildMISPAttribute.py", line 418, in parseIndicators
    for intent in pkg.stix_header.package_intents:
AttributeError: 'NoneType' object has no attribute 'package_intents'

[chrisinmtown]

You posted about a script in the tools area, but that only seems to do a conversion, it does not upload to the MISP server, and I think you are correct in saying it's limited to stix version 1.

But the issue title here says "import". I see that PyMISP provides method upload_stix and the version parameter in that method defaults to "2". Did you try calling that method?

[Rafiot]

@chrisr3d is our master of all stix related things.

IIRC, this piece of code is deprecated and should be removed or completely rewritten. And it kind of supports STIX 1, if the STIX 1 file is not too much of a disaster. The recommended way to import STIX files is to use the web interface, and hope the creator of the report was not too creative in the interpretation of the standard.

[chrisr3d]

The web interface works indeed for such kind of data to import.

Alternatively, the /events/upload_stix/2 endpoint works as well, what you need to do is simply to copy the content of your STIX file:

Once you run the query you also get the python equivalent using PyMISP if you want to automate on multiple STIX files

[Rafiot]

For reference, here is the documentation for PyMISP STIX upload: https://pymisp.readthedocs.io/en/latest/modules.html#pymisp.PyMISP.upload_stix

[chrisinmtown]

@tahaconfiant if your question is answered, would please close this issue?

[tahaconfiant]

nearly there, running the suggested script:

x@misp:/var/www/MISP/tools/ingest_stix$
 sudo -H -u www-data /var/www/MISP/venv/bin/python3 ingest_stix.py --version 2 --path stix2.json

gave me a

<Response [403]>
Error with the ingestion of the following file: stix2.json

checking error logs :

tail: /var/www/MISP/app/tmp/logs/exec-errors.log: file truncated
Traceback (most recent call last):
  File "/var/www/MISP/app/files/scripts/stix2/stix2misp.py", line 2061, in <module>
    main(sys.argv)
  File "/var/www/MISP/app/files/scripts/stix2/stix2misp.py", line 2053, in main
    event = stix2.parse(f.read(), allow_custom=True, interoperability=True)
TypeError: parse() got an unexpected keyword argument 'interoperability'

my packages versions , I think I have the last version of everything stix related, just upgraded them all:

Error with the ingestion of the following file: stix2.json
x@misp:/var/www/MISP/tools/ingest_stix$ sudo -H -u www-data /var/www/MISP/venv/bin/pip3 list  | grep stix
stix                   1.2.0.11
stix2                  2.1.0
stix2-patterns         1.3.2

[brlogan]

Hey @tahaconfiant, maybe you figured this out by now, but MISP uses a custom forked version of the cti-python-stix2 library which includes the interoperability parameter. https://github.com/MISP/cti-python-stix2/blob/main/stix2/parsing.py

You can read more about the decision here: https://github.com/oasis-open/cti-python-stix2/pull/235