CrowdStrike connector work jobs increasing at a steady rate due to "Cant upsert entity. Too many entities resolved"

[brett-fitz]

Description

CrowdStrike's connector jobs are increasing at a steady rate after an issue came up with one of the STIX bundles:

Message

{'name': 'UNSUPPORTED_ERROR', 'error_message': 'Cant upsert entity. Too many entities resolved', 'http_status': 500, 'genre': 'BUSINESS', 'entityIds': ['intrusion-set--9a4f2db4-ef15-590b-9006-3caf2631269b', 'intrusion-set--e39e6fa1-6a32-5397-b010-a6f1e73e9929']}

Source

{"type": "intrusion-set", "spec_version": "2.1", "id": "intrusion-set--03ca672d-1052-5899-bccd-6ba1c75ceaaf", "created_by_ref": "identity--f29f12ba-3980-5642-9b3e-d11e9b296aed", "created": "2024-06-06T21:09:57.215776Z", "modified": "2024-06-06T21:09:57.215776Z", "name": "BERSERK BEAR", "aliases": ["BERSERKBEAR"], "external_references": [{"source_name": "CrowdStrike", "url": "https://falcon.crowdstrike.com/intelligence/actors/berserk-bear/", "external_id": "1809"}], "object_marking_refs": ["marking-definition--f88d31f6-486f-44da-b317-01333bde0b82"], "nb_deps": 3, "x_opencti_stix_ids": null, "x_opencti_granted_refs": null, "x_opencti_workflow_id": null}

Environment

1. OS (where OpenCTI server runs): AWS ECS
2. OpenCTI version: 6.1.10
3. OpenCTI client: python
4. Other environment details:

Reproducible Steps

Steps to create the smallest reproducible scenario:

1. Run the CrowdStrike connector
2. Wait

Expected Output

Connector should just import the new entities and perhaps skip the bundle that is error'ing out.

Actual Output

As we can see once the error pops up the work starts increasing likely due to this failure.

Additional information

Screenshots (optional)

• Screenshots above

[Megafredo]

Hi @brett-fitz, if you get this error: 'UNSUPPORTED_ERROR' => 'Cant upsert entity. Too many entities resolved' then this is expected behavior and the problem lies in the names and aliases of your intrusion sets.

In fact, if you have two different intrusion sets corresponding to this set of names and/or aliases, then OpenCTI is not able to determine which set of intrusion to select and then attach the knowledge.

To resolve this issue, simply merge your intrusion sets or edit the alias distribution in your OpenCTI platform to match the CrowdStrike data source.

When manually creating an intrusion set, you can see the number of duplicates for example:

[brett-fitz]

@Megafredo Would we expect this to occur with two different entity types?

[Megafredo]

@brett-fitz If the entities are unrelated by name or alias, then yes, there is indeed a problem. We will investigate. Thanks

[brett-fitz]

@Megafredo Ok thanks. I was able to fix 1/2 - "PUNK SPIDER" (CrowdStrike) / Akira (MITRE) but BERSERK BEAR is still an issue. Is there any way of getting around this to fix the connector's jobs from getting exponentially larger?

[brett-fitz]

FYSA: There is no alias on the malware

The only connection I could find (outside of relationship with Dragonfly which has an alias with BERSERK BEAR) is this external reference on Havex:

[brett-fitz]

More screenshots:

Note: I did merge BERSERK BEAR with Dragonfly and made Dragonfly the primary entity.

[brett-fitz]

Hey @Megafredo any update on the above? Number of operations for us has gotten do 60k and steadily increasing (its ingesting all data since the last reported time in the state). Is there a stop gap fix for this?

[romain-filigran]

I close the issue as discussed