search

OpenCTI-Platform / opencti

Open Cyber Threat Intelligence Platform
https://opencti.io
Other
5.18k stars 820 forks source link

import_stix2.py fails with UTF-8 values #2944

Open kmcmahon1959 opened 1 year ago

kmcmahon1959 commented 1 year ago

Please replace every line in curly brackets { like this } with an appropriate answer, and remove this line.

Description

Loading a STIX2 bundle using the import_stix2.py script fails with "ERROR:root:Observable of type Domain-Name is not correctly formatted."

Environment

  1. Ubuntu 20.04
  2. OpenCTI 5.5.4
  3. Other environment details:

Reproducible Steps

Steps to create the smallest reproducible scenario:

  1. Using export_stix2.py on system1 export domain-name--f7ff12d6-c9df-5501-b64c-97e923ec1a09
  2. Move file to system2
  3. Run import_stix2.py for domain-name--f7ff12d6-c9df-5501-b64c-97e923ec1a09

Expected Output

Expecting the bundle to be loaded as it exists on system 1.

Actual Output

ERROR:root:Observable of type Domain-Name is not correctly formatted. Traceback (most recent call last): File "/opt/opencti-highside-sync/./sync-data/src/import-stix2-file.py", line 15, in opencti_api_client.stix2.import_bundle_from_file(file_to_import, True) File "/usr/local/lib/python3.10/site-packages/pycti/utils/opencti_stix2.py", line 185, in import_bundle_from_file return self.import_bundle(data, update, types) File "/usr/local/lib/python3.10/site-packages/pycti/utils/opencti_stix2.py", line 2026, in import_bundle self.import_observable(item, update, types) File "/usr/local/lib/python3.10/site-packages/pycti/utils/opencti_stix2.py", line 822, in import_observable stix_observable_result = self.opencti.stix_cyber_observable.create( File "/usr/local/lib/python3.10/site-packages/pycti/entities/opencti_stix_cyber_observable.py", line 1163, in create result = self.opencti.query(query, input_variables) File "/usr/local/lib/python3.10/site-packages/pycti/api/opencti_api_client.py", line 332, in query raise ValueError( ValueError: {'name': 'FunctionalError', 'message': 'Observable of type Domain-Name is not correctly formatted.'} Killed

kmcmahon1959 commented 1 year ago

Oops, here the actual bundle with the domain-name:
{ "type": "bundle", "id": "bundle--af9da8bc-6702-4ccf-b1bd-b65ed108e609", "objects": [ { "id": "identity--0a225431-f1d7-5e77-99fc-6f5d392b92d9", "spec_version": "2.1", "identity_class": "organization", "name": "Mandiant", "description": "Mandiant is recognized by enterprises, governments and law enforcement agencies worldwide as the market leader in threat intelligence and expertise gained on the frontlines of cyber security. ", "created": "2022-09-23T14:44:30.951Z", "modified": "2022-09-23T14:44:30.951Z", "x_opencti_id": "32d191c4-c110-419d-9f96-4995f1dd5b4a", "type": "identity" }, { "type": "marking-definition", "spec_version": "2.1", "id": "marking-definition--f88d31f6-486f-44da-b317-01333bde0b82", "created": "2017-01-20T00:00:00.000Z", "definition_type": "tlp", "name": "TLP:AMBER", "definition": { "tlp": "amber" } }, { "id": "domain-name--f7ff12d6-c9df-5501-b64c-97e923ec1a09", "spec_version": "2.1", "x_opencti_description": "Simple observable of indicator {домен.ru}", "x_opencti_score": 50, "value": "домен.ru", "x_opencti_id": "094396df-bd9e-48ad-b5b0-076eea476664", "type": "domain-name", "created_by_ref": "identity--0a225431-f1d7-5e77-99fc-6f5d392b92d9", "object_marking_refs": [ "marking-definition--f88d31f6-486f-44da-b317-01333bde0b82" ] } ] }

SamuelHassine commented 1 year ago

@team: Improve the domain name regular expression to support UTF-8 chars.