OpenCTI-Platform / opencti

Open Cyber Threat Intelligence Platform
https://opencti.io
Other
6.32k stars 932 forks source link

Report Generation : An unknown error occurred [UPDATED] #2232

Closed stevehenderson closed 2 years ago

stevehenderson commented 2 years ago

Description

When trying to create a report manually (Via Analysis Tab), I received the following:

image

An unknown error occurred. Please contact your administrator or the OpenCTI maintainers.

Both issues seem to be related to how OpenCTI is provisioning reports.

Environment

  1. OS (where OpenCTI server runs):
Distributor ID: Ubuntu
Description:    Ubuntu 22.04 LTS
Release:    22.04
Codename:   jammy
  1. OpenCTI version: { e.g. OpenCTI 1.0.2 }
opencti-release-5.3.7.tar.gz 
  1. OpenCTI client:

    • Chrome: Version 102.0.5005.115 (Official Build) (64-bit)
  2. Other environment details:

    • Installed behind Nginx reverse proxy with SSL termination
    • Node version: v16.16.0
    • Yarn version: 1.22.19
    • Elastic 7.17.5
    • Python 3.9.12
    • Running the Auth0Strategy authentication provider
      • Not sure its related yet, but my report problems started once I got Auth0 working

Reproducible Steps

Steps to create the smallest reproducible scenario:

Case 1

  1. Login as an Auth0 authenticated user with Admin permissions
  2. Visit Activities > Analysis
  3. Click the Plus Sign icon in lower right
  4. Error:
opencti-3343EXMJ.js:7 RangeError: Format string contains an unescaped latin alphabet character `n`
    at opencti-3343EXMJ.js:7926:44416
    at Array.map (<anonymous>)
    at Xhn (opencti-3343EXMJ.js:7926:44173)
    at n.formatByString (opencti-3343EXMJ.js:7926:75561)
    at V9e (opencti-3343EXMJ.js:8125:85841)
    at opencti-3343EXMJ.js:8125:95416
    at h5n (opencti-3343EXMJ.js:5:53950)
    at fnt (opencti-3343EXMJ.js:5:62180)
    at art (opencti-3343EXMJ.js:9:10957)
    at nrt (opencti-3343EXMJ.js:9:975)

Post call:

curl 'https://opencti.teleseer.com/graphql' \
  -H 'authority: opencti.[redacted].com' \
  -H 'accept: */*' \
  -H 'accept-language: en-US,en;q=0.9' \
  -H 'cache-control: no-cache' \
  -H 'content-type: application/json' \
  -H '[redacted]' \
  -H 'origin: [redacted]' \
  -H 'pragma: no-cache' \
  -H 'referer: https://opencti.[redacted].com/dashboard/analysis/reports?sortBy=name&orderAsc=true&filters=%7B%22report_types%22%3A%5B%7B%22id%22%3A%22threat-report%22%2C%22value%22%3A%22threat-report%22%7D%5D%7D&deSelectedElements=null' \
  -H 'sec-ch-ua: " Not A;Brand";v="99", "Chromium";v="102", "Google Chrome";v="102"' \
  -H 'sec-ch-ua-mobile: ?0' \
  -H 'sec-ch-ua-platform: "Linux"' \
  -H 'sec-fetch-dest: empty' \
  -H 'sec-fetch-mode: cors' \
  -H 'sec-fetch-site: same-origin' \
  -H 'user-agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36' \
  --data-raw $'{"id":"AttributesLinesAttributesQuery","query":"query AttributesLinesAttributesQuery(\\n  $key: String\u0021\\n) {\\n  runtimeAttributes(attributeName: $key) {\\n    edges {\\n      node {\\n        id\\n        key\\n        value\\n      }\\n    }\\n  }\\n}\\n","variables":{"key":"report_types"}}' \
  --compressed

Expected Output

I expect to see the Create a report dialog (demo site shown):

image

Actual Output

image

Additional information

I think something is wrong with Report ingestion. Although I ingested some reports from AlienVault early on, now I'm seeing a bunch of this in server log:

{
    "category":"APP",
    "error":{
        "data":{"category":"business","http_status":404,"input":{"confidence":15,"created":"2022-02-28T15:51:42.427Z","createdBy":"identity--e52b2fa3-2af0-5e53-ad38-17d54b3d61cb","description":"A threat report published by Symantec in October 2021 recently caught Fortinet's attention. It discusses an unknown threat actor conducting an espionage campaign in Southeast Asia using a new custom malware arsenal. What piqued Fortinet's curiosity most was the mention of a DLL payload loaded from the registry that had yet to be discovered.\n\nThe module is stored as a compressed blob with a custom header in the registry. It is never written to disk, rendering it unlikely to appear in datasets like VirusTotal.\n\nFortinet's have now uncovered a sample of the module and a plethora of components and variants dating as far back as 2017. They have observed the progression of the development of this malware throughout the years. Over time, custom code was added, components were upgraded, capabilities expanded, the code became neater, and modularity increased.\n\nThis blog examines the different components of this malware and their progression over time, thereby mapping the evolution of the Soul malware framework.","externalReferences":["2728e53b-b26a-4299-afa2-c8801cf622d2","78f2e77b-59ab-4bcf-8c43-84a1e124a1ca"],"lang":null,"modified":"2022-03-30T00:00:10.458Z","name":"The Hunt for the Lost Soul: Unraveling the Evolution of the SoulSearcher Malware","objectLabel":["6f3e0af5-0d83-4fee-b915-fa7b78faa8e8"],"objectMarking":["marking-definition--613f2e26-407d-48c7-9eca-b8e91df99dc9"],"objects":["relationship--71a92fd8-cd89-598e-b75b-8a890a5b58c8","relationship--1c795f16-b5d1-5acd-86d0-3e2b03fee6b9","relationship--82b238f5-9236-5dae-907e-10125eb5cea8","relationship--e3f67ee3-929c-545c-80df-f7761ba4df1d","relationship--6333517c-950b-5646-b24f-e7adf2890719","relationship--b456befb-59db-5103-8538-b37784108c6a","relationship--9c0dae4b-d2cb-5f70-a234-fc58b355d155","relationship--4e6e43d5-d3cd-58b5-a0ea-ec33ee97a684","relationship--69a97aba-2906-5f47-8ca0-20080d0ca224","relationship--dc803c26-ffcc-5f5c-be77-2baf86b59866","relationship--93f66577-8c11-5feb-96fe-5ee087472a3e","relationship--eac09845-f5af-50e0-a815-052cbcff8277","relationship--402ad15f-7a90-5e73-aa18-14295313465f","relationship--798ece00-c572-5bb8-b18a-77e99dcf9a79","relationship--31e9cde8-319a-58cc-9e16-64e82af208bb","relationship--3b24c699-1ecc-5b2c-bfa3-537d4f9b9c9b","relationship--2ca2fd1f-b90d-5b3e-9e18-366945adc919","relationship--effbfe09-c445-57a2-8977-6dc2ff80304e","relationship--bdd9a2b8-96ea-5cd5-a976-902f65e04ef6","relationship--0be6e511-9520-5a97-825e-b22f36c6776b","relationship--fcf58480-7ec7-509f-aa49-3d84cfd56398","relationship--d4aecdc3-3ff1-51df-973a-c2cd78c06494","relationship--d9f43092-f217-562c-b334-d3300bafc642","relationship--c7ffef7d-1bfc-50a5-8627-577067c27899","relationship--4720a349-5cfc-5d14-87fc-2863fb9325b8","relationship--641858b5-cf30-50f1-8bcd-769740c9d096","relationship--026558c7-4438-5889-a87d-64fec3594bd6","relationship--de411cdd-c4a0-546e-b6f7-17655b3006ed","relationship--8816f424-3fe9-5f40-b173-fbb8856a3ed3","relationship--ecbbf706-8a24-58ab-83cd-db0ca255723d","relationship--3f538174-93fc-532d-bc8d-282cc021afa1","relationship--ff0430d6-e9e1-5ff1-97ac-9eaff5c220a2","relationship--f4efa466-e0cb-5957-be01-361c05c63aee","relationship--711f9599-9651-531a-9086-58730f024d2a","relationship--35ef3ad1-5245-5296-af30-366dd1f25634","relationship--27bb2618-a2b9-57c2-adda-a5a13bb6b686","relationship--254b44bc-8869-5147-aa2d-3f656b08106e","relationship--969a3f39-d4ea-510a-a9d0-940eae5252b3","relationship--bad293af-fd58-5859-8a12-f20be90eb30b","relationship--82bf1e74-c03a-544e-a821-dbb032c5e8ff","relationship--63c9a589-7bd6-5e2b-8a85-e88e49cbdfd4","relationship--92c1fd5a-0844-5cf3-9218-169227578ad9","relationship--477e332f-b53d-50eb-8a53-585c1cfc82a1","relationship--b783ee09-6f1e-54ab-b916-0dcaaf7b8f3b","relationship--c8a00609-6b96-544e-8527-49637df170a6","relationship--b4d79cc3-e451-5b45-86d7-f6b150d7a7e5","relationship--3d6bdb29-699a-57f6-a9b7-93185814827a","relationship--d27ff279-8db4-59d2-b1fb-6bee1b2f6ca8","relationship--0f5594af-061d-5448-bccf-3fdea66a2a6f","relationship--38eb759a-6801-5951-80fa-f06566f4c3ff","relationship--c12d1dd5-a9a3-57b1-a887-fedb022aa296","relationship--47684b86-f9da-540d-a8b6-a64f70d07783","relationship--7b87132a-6046-5d5f-a515-01ac2f6d4a52","relationship--034e0edc-bf67-59a5-b0a4-6f6227fcd144","relationship--0e312285-86ea-5bdd-a069-fd437af4f660","relationship--5c7dd614-f85a-5841-95dd-22abd148a0d9","relationship--4df430e9-f758-5c8f-acf0-60d43554922e","relationship--8ee0a482-5c5b-5f90-9e56-3a2f8a1153b4","relationship--bf15c7a8-09f0-5269-8992-496377c8e45c","relationship--0d945661-4046-5aaa-97ce-51570e5205b2","relationship--af85be77-8937-5fb3-bdea-5428e9f52ebd","relationship--db27e524-b215-5c99-a6ee-7b935101d6aa","relationship--ce0884f2-18a8-5e5f-b13c-fd6e19a81f49","relationship--34e5d62e-8ce4-5889-9144-293e30b2506f","relationship--8b5ea2f0-5b1c-5236-b52a-325d6d438eae","relationship--c181fb4c-9fda-5e75-84e5-0ddbafddb71c","relationship--8befb6b6-4329-5d44-b9b4-08cf0271a6e4","relationship--92de39a6-dc34-5a11-928f-1f99244aa075","relationship--f361e8ea-75c2-54fb-b835-44012dd9ea8e","relationship--1929d0c8-e46c-5087-bef4-57d1e49cce01","relationship--f1c47e1f-8117-5c99-82fe-6442fca05bbd","relationship--b9f97195-a234-5aa9-83f2-f7203496a2e5","relationship--67140f7f-779f-565d-8971-39c61ce8ebf0","relationship--b5beed93-bcfa-5bb6-a383-0033a2ebf82c","relationship--65e994fa-521a-5d00-b798-d0e89d2394aa","relationship--76f69f52-c535-515d-a9d5-e4ad01b4cb8f","relationship--533181c1-9347-57b5-b8c8-b40dff4b0714","relationship--7cca8c86-9d61-5a68-8d26-ac47c8a51fee","relationship--2391b9ba-7d59-5763-a6da-764d3d5df51d","relationship--627e1f3f-2031-560f-b614-f2c2f1bae78f","relationship--934340c1-01c1-526e-ac41-df86be9566e2","relationship--b433a1df-1162-5209-9cc8-3d367d714c47","relationship--b6b94075-ada3-50a1-958f-7ef5b497d279","relationship--d23405d0-f21e-55b6-a65b-21d3b24b5f21","relationship--19f661e4-07fa-5be2-b841-25a4fa97e728","relationship--309ff820-2ccc-574b-aa89-3f1c68ec5579","relationship--d020b74d-50c4-5390-8198-e0e474e9050b","relationship--ede3e0ae-ee64-5436-9a7e-9e9aae467c2f","relationship--7de3cbc1-26b3-5b72-8c73-653cae4878e2","relationship--667d5375-94ad-5f96-89ab-5b0e7bd2a372","relationship--3833c3d1-1f4b-5c95-9325-4be23f6dccbe","relationship--0ac678ff-c63d-5351-9828-4d5417b1a3fb","relationship--b04237f1-ec16-5101-a9cc-25b71cde5560","relationship--53608d86-dd75-5588-bd99-1edc31349bd3","relationship--d9e6d9a1-8d2c-5e11-aeb2-596a782eac0b","relationship--e61d84b1-2669-5688-aae9-876c87282c62","relationship--7de622c5-14b4-5729-8c84-80b14909dbfd","relationship--644a404d-92a4-57a0-89ba-e4e6213a8957","relationship--a492fa00-512d-5293-b13c-9140a2da9002","relationship--b7d4ef61-c938-58c3-a835-0eddf9e5cb72","relationship--d1bb04d5-7af3-5905-ac47-3236ceec3f2d","relationship--abb70bd6-0770-513c-a23d-bedd6c549d27","relationship--805393f8-19ed-552d-9206-8363f3ad8b5e","relationship--a6989695-c840-526b-acb3-11907d31c221","relationship--ac251435-c9f9-599b-b609-fc43491d941f","relationship--dd2a706a-4af9-5b30-8a3d-8af6980c422c","relationship--1094e8e6-0f2f-57ec-8cf2-2ac77ee7ebfb","relationship--ed44a641-2154-59cc-bc20-6318e4587f70","relationship--e43afab2-c11b-5776-bd05-a45e990c1d36","relationship--4bafaba6-7b33-59b6-9548-c5eb8d9bd7ad","relationship--24864d6d-6ba6-57f3-9bdd-0736f9a333e7","relationship--bb8cff8a-df85-5224-b10a-6f7332841790","relationship--74e9262d-3841-51f8-8f7d-7f227e1ba0a9","relationship--4b858316-05e7-50f8-99e5-1df9e64eae1f","relationship--9aaf3793-1632-5dfd-aa2f-d6b92f677de4","relationship--8f0fe781-90c0-5e2a-8a03-7d3552544cf0","relationship--2b9f21a6-4335-50be-b40b-712e4d1cf4c8","relationship--79f09278-389e-5c82-8872-5e399a2c266f","relationship--cea76a49-80d4-5d73-bf61-0cadf8bec74d","relationship--acc92857-7933-5e73-bf04-a13d62147e70","relationship--18ff188d-bd54-59dc-bf5b-108d631df73a","relationship--253132bf-9b55-5a96-ab2f-6d49e389450f","relationship--909ae324-8c22-5804-8457-2130d5e2a6c3","relationship--662d6448-8e00-5b98-8919-be0188d0a1d0","relationship--0898f99a-7a27-5c9e-bc38-68326378418e","relationship--6e31d1f8-d3da-59b8-997a-eb737443372b","relationship--d13b555e-589e-52d8-a6a3-c3599f88dafc","relationship--629d76f5-c798-540f-8ca6-5339e9a34d4e","relationship--c04f7eae-b177-5d06-adab-f3efaccbbed2","relationship--82d27d5c-5eb5-53e4-b2a1-bc0f1ccbfb65","relationship--1d955d8b-f5a7-50da-96b3-0ccd2c04ba85","relationship--640d3e30-258c-5334-b152-acf57758250a","relationship--6bb03ee7-844e-53e8-bde8-8c9cc85b7b1f","relationship--0a582de5-1227-5337-9ccb-67150d51d877","relationship--14e46ba0-7897-5d94-a2fd-3d25d651ee59","relationship--7845be85-eb57-5b66-b984-c3a20e5307c2","relationship--f1fb18ad-0f5c-578c-b69d-d79f51a41f8f","relationship--3f64f4a0-abad-5bbd-8f71-1094fbfbd2b8","relationship--14d130a8-ab62-5596-aae4-7c445a2777b7","relationship--7070dd12-b535-540b-a001-a6a6eed4977a","relationship--aeedcd23-a7f0-5597-82bc-b6857e5e41a0"],"published":"2022-02-28T15:51:42.427Z","report_types":["threat-report"],"revoked":null,"stix_id":"report--b1ed5717-8879-512e-98ae-72b14bdca1ef","update":false,"x_opencti_stix_ids":null},
        "reason":"Missing reference to handle creation",
        "unresolvedIds":["relationship--1d955d8b-f5a7-50da-96b3-0ccd2c04ba85","relationship--0a582de5-1227-5337-9ccb-67150d51d877","relationship--7845be85-eb57-5b66-b984-c3a20e5307c2","relationship--f1fb18ad-0f5c-578c-b69d-d79f51a41f8f"]},
        "stacktrace":[
            "MissingReferenceError: Element not found","at error (/home/redacted/opencti/build/src/config/errors.js:8:10)",
            "at MissingReferenceError (/home/redacted/opencti/build/src/config/errors.js:93:48)",
            "at inputResolveRefs (/home/redacted/opencti/build/src/database/middleware.js:686:11)",
            "at createEntityRaw (/home/redacted/opencti/build/src/database/middleware.js:2827:25)",
            "at createEntity (/home/redacted/opencti/build/src/database/middleware.js:2955:7)",
            "at addReport (/home/redacted/opencti/build/src/domain/report.js:131:19)"]},
            "inner_relation_creation":1,
            "level":"warn",
            "message":"API Call",
            "operation":"ReportAdd",
            "operation_query":
            "mutation ReportAdd($input:ReportAddInput){reportAdd(input:$input){id standard_id entity_type parent_types}}",
                        "size":40699,
                        "time":1405,
                        "timestamp":"2022-07-18T02:44:43.425Z",
                        "type":"WRITE_ERROR",
                        "user":{"applicant_id":"[redacted]",
                        "call_retry_number":"1",
                        "ip":"::1",
                        "user_id":"[redacted]"},
                        "version":"5.3.7"
}

These seem to be spawning for every create report event...I see them with the UploadFile connector and others.

Update

stevehenderson commented 2 years ago

I was able to resolve this issue. My Auth0 admin user didn't not have a default language. Once I set this, it seemed to fix the Report creation issue. Closing this out.

nicocti commented 2 years ago

Hello, same problem here. Thanks @stevehenderson for the follow up. May be this should be documented ?

SamuelHassine commented 2 years ago

@stevehenderson, @nicocti: https://github.com/OpenCTI-Platform/opencti/issues/2227.