OpenCTI-Platform / opencti

Open Cyber Threat Intelligence Platform
https://opencti.io
Other
5.2k stars 821 forks source link

Removing an alias reset the entire list of aliases #7022

Open labo-flg opened 1 month ago

labo-flg commented 1 month ago

Description

When you remove an alias from an object, the new list of aliases is passed to a field patch mutation, that reset entirely the list of aliases.

Doing so, each alias of the list is checked against the live database to see if it collides with an existing object. It makes sense as we theoretically change the whole list, but in the case of a removal, this should not be necessary.

An object can have a very long list of aliases (happens in prod, ~200 aliases on a malware), so this step can in fact consume a lot of processing and queries.

NB: In one case, this lead to errors in production (an old database with somehow colliding aliases = check fails on removing any alias). This is an edge-case that should not be possible as we have proper check in place now.

Environment

ALL

Reproducible Steps

Create many aliases for a malware. Try to delete one of them checkout the query payload: it's the whole list passed as a field patch.

Additional information

We could optimize this process by calling a field patch with object_path at the right index, and remove operation. In that case, no check for duplicate necessary.

Incidentally, it would solve our edge-case as removing any alias will always be possible, and manually solving the existing alias duplicates would be ok.

richard-julien commented 1 month ago

That will not solved the underlying problem, need to discuss about that

labo-flg commented 1 month ago

Solution to investigate:

middleware:1838 this is where the alias are checked.

If we can add the ability to patch add/remove multiple field (not object) then we can try to check alias only on add/replace, and not on remove.

Investigate how it's done in upsert, theoretically we do not replace there.

labo-flg commented 3 weeks ago

For the record, we could also enable field patching with non-object attributes ; this is another story in terms of complexity.