search

TheHive-Project / TheHive

TheHive: a Scalable, Open Source and Free Security Incident Response Platform
https://thehive-project.org
GNU Affero General Public License v3.0
3.41k stars 623 forks source link

[Bug] Merge alert Into case [duplication of objects] #2064

Open Gigazo1d opened 3 years ago

Gigazo1d commented 3 years ago

Request Type

Bug

Work Environment

Question Answer
OS version (server) Ubuntu 20.04.2
OS version (client) Win 10 Pro
Virtualized Env. True
Dedicated RAM 3 GB
vCPU 2
TheHive version / git hash 4.1.4-1
Package Type From source
Database Cassandra
Index type Elasticsearch
Attachments storage Local
Browser type & version Chrome 90.0.4430.212

Problem Description

Hello! Problems encountered when merging incidents into a case. I will say right away in version 3.4 there were no problems. From SIEM incident_id are unloaded into the Hive ['object']['customFields']['siem_inc_id']['string']. At the moment of merging 3+ incidents, the object of three incidents siem_inc_id is filled with the id of the last attached incident. What we have: 3 incidents, alerts merge to the case, webhooks give 3 siem_inc_id of the last incident in the list instead of three different ones. As a result of this bug, only one incident is updated in SIEM. If the merge is one incident, then all is well.

Steps to Reproduce

  1. Open an alert
  2. Try to merge 3+ incident it using the "Merge Into Case" button

Complementary information

For test use default Flask config: 

from flask import Flask, request
import json

app = Flask(__name__)

@app.route('/',methods=['POST'])
def foo():
   data = json.loads(request.data)
   print(json.dumps(data, indent=4))
   return "OK"

if __name__ == '__main__':
   app.run()

Logs flask server:

Create Case:

{
    "operation": "create",
    "details": {
        "_id": "~**206028968**", To this case_id we will merge
        "_type": "Case",
        "_createdBy": "test@mail.ru",
        "_createdAt": 1623060982047,
        "number": 78,
        "title": "\u041f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u044c root \u0441 UID 0 \u043f\u0440\u0438 \u043f\u043e\u043c\u043e\u0449\u0438 \u0443\u0442\u0438\u043b\u0438\u0442\u044b sudo \u0432\u044b\u043f\u043e\u043b\u043d\u0438\u043b \u043a\u043e\u043c\u0430\u043d\u0434\u0443 sudo bash /tmp/61a7-3f05-7851-586d \u0434\u043b\u044f \u043f\u043e\u0432\u044b\u0448\u0435\u043d\u0438\u044f \u043f\u0440\u0438\u0432\u0438\u043b\u0435\u0433\u0438\u0439 \u043d\u0430 \u0443\u0437\u043b\u0435 132.29.61.460",
        "description": "#### C\u0441\u044b\u043b\u043a\u0430 INC-15830 \u0432 nts/view/126c5428-e708-4afd-a4b3-2c3eac7323cc?groupId=all_incidents&tabName=tasks**\n\n\n",
        "severity": 3,
        "startDate": 1623060982034,
        "tags": [
            "Detect_PrivEscalation_via_GTFOBINS"
        ],
        "flag": false,
        "tlp": 3,
        "pap": 2,
        "status": "Open",
        "assignee": "test@mail.ru",
        "customFields": {
            "category": {
                "string": "Undefined",
                "order": 0
            },
            "type": {
                "string": "Undefined",
                "order": 1
            },
            "status": {
                "string": "\u0417\u0430\u043a\u0440\u044b\u0442",
                "order": 2
            },
            "siem_inc_id": {
                "string": "126c5428-e708-4afd-a4b3-2c3eac7323cc", 
                "order": 3
            },
            "rule": {
                "string": "Detect_PrivEscalation_via_GTFOBINS",
                "order": 4
            },
            "time-incident": {
                "date": 1623060006000,
                "order": 5
            },
            "false": {
                "string": null,
                "order": 6
            }
        },
        "extraData": {},
        "fromAlert": {
            "_id": "~329920656",
            "type": "Undefined",
            "source": "TEST SIEM",
            "sourceRef": "INC-15830"
        }
    },
    "objectType": "case",
    "objectId": "~206028968",
    "base": true,
    "startDate": 1623060982119,
    "rootId": "~206028968",
    "requestId": "998486afe9453169:-65c6db37:179e53a9acf:-8000:1470",
    "object": {
        "_id": "~206028968",
        "id": "~206028968",
        "createdBy": "test@mail.ru",
        "updatedBy": null,
        "createdAt": 1623060982047,
        "updatedAt": null,
        "_type": "case",
        "caseId": 78,
        "title": "\u041f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u044c root \u0441 UID 0 \u043f\u0440\u0438 \u043f\u043e\u043c\u043e\u0449\u0438 \u0443\u0442\u0438\u043b\u0438\u0442\u044b sudo \u0432\u044b\u043f\u043e\u043b\u043d\u0438\u043b \u043a\u043e\u043c\u0430\u043d\u0434\u0443 sudo bash /tmp/61a7-3f05-7851-586d \u0434\u043b\u044f \u043f\u043e\u0432\u044b\u0448\u0435\u043d\u0438\u044f \u043f\u0440\u0438\u0432\u0438\u043b\u0435\u0433\u0438\u0439 \u043d\u0430 \u0443\u0437\u043b\u0435 171.34.22.45",
        "description": "#### C\u0441\u044b\u043b\u043a\u0430 INC-15830 dents/view/126c5428-e708-4afd-a4b3-2c3eac7323cc?groupId=all_incidents&tabName=tasks**\n\n\n",
        "severity": 3,
        "startDate": 1623060982034,
        "endDate": null,
        "impactStatus": null,
        "resolutionStatus": null,
        "tags": [
            "Detect_PrivEscalation_via_GTFOBINS"
        ],
        "flag": false,
        "tlp": 3,
        "pap": 2,
        "status": "Open",
        "summary": null,
        "owner": "test@mail.ru",
        "customFields": {
            "time-incident": {
                "date": 1623060006000,
                "order": 5
            },
            "rule": {
                "string": "Detect_PrivEscalation_via_GTFOBINS",
                "order": 4
            },
            "false": {
                "string": null,
                "order": 6
            },
            "status": {
                "string": "\u0417\u0430\u043a\u0440\u044b\u0442",
                "order": 2
            },
            "type": {
                "string": "Undefined",
                "order": 1
            },
            "category": {
                "string": "Undefined",
                "order": 0
            },
            "siem_inc_id": {
                "string": "126c5428-e708-4afd-a4b3-2c3eac7323cc",
                "order": 3
            }
        },
        "stats": {},
        "permissions": []
    },
    "organisationId": "~16464",
    "organisation": "Test"
}127.0.0.1 - - [07/Jun/2021 10:16:22] "POST / HTTP/1.1" 200 -
127.0.0.1 - - [07/Jun/2021 10:16:22] "POST / HTTP/1.1" 200 -

{
    "operation": "update",
    "details": {
        "caseId": "~206028968"
    },
    "objectType": "alert",
    "objectId": "~329920656",
    "base": false,
    "startDate": 1623060982118,
    "rootId": "~206028968",
    "requestId": "998486afe9453169:-65c6db37:179e53a9acf:-8000:1470",
    "object": {
        "_id": "~206028968",
        "id": "~206028968",
        "createdBy": "test@mail.ru",
        "updatedBy": null,
        "createdAt": 1623060982047,
        "updatedAt": null,
        "_type": "case",
        "caseId": 78,
        "title": "\u041f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u044c root \u0441 UID 0 \u043f\u0440\u0438 \u043f\u043e\u043c\u043e\u0449\u0438 \u0443\u0442\u0438\u043b\u0438\u0442\u044b sudo \u0432\u044b\u043f\u043e\u043b\u043d\u0438\u043b \u043a\u043e\u043c\u0430\u043d\u0434\u0443 sudo bash /tmp/61a7-3f05-7851-586d \u0434\u043b\u044f \u043f\u043e\u0432\u044b\u0448\u0435\u043d\u0438\u044f \u043f\u0440\u0438\u0432\u0438\u043b\u0435\u0433\u0438\u0439 \u043d\u0430 \u0443\u0437\u043b\u0435 132.59.51.145",
        "description": "#### nt/incidents/view/126c5428-e708-4afd-a4b3-2c3eac7323cc?groupId=all_incidents&tabName=tasks**\n\n\n",
        "severity": 3,
        "startDate": 1623060982034,
        "endDate": null,
        "impactStatus": null,
        "resolutionStatus": null,
        "tags": [
            "Detect_PrivEscalation_via_GTFOBINS"
        ],
        "flag": false,
        "tlp": 3,
        "pap": 2,
        "status": "Open",
        "summary": null,
        "owner": "test@mail.ru",
        "customFields": {
            "time-incident": {
                "date": 1623060006000,
                "order": 5
            },
            "rule": {
                "string": "Detect_PrivEscalation_via_GTFOBINS",
                "order": 4
            },
            "false": {
                "string": null,
                "order": 6
            },
            "status": {
                "string": "\u0417\u0430\u043a\u0440\u044b\u0442",
                "order": 2
            },
            "type": {
                "string": "Undefined",
                "order": 1
            },
            "category": {
                "string": "Undefined",
                "order": 0
            },
            "siem_inc_id": {
                "string": "126c5428-e708-4afd-a4b3-2c3eac7323cc",
                "order": 3
            }
        },
        "stats": {},
        "permissions": []
    },
    "organisationId": "~16464",
    "organisation": "Test"
}

Updating the last and key alert and creating a case from it:

{
    "operation": "update",
    "details": {
        "customFields": {
            "time-incident": {
                "date": 1623060001000,
                "order": 4
            },
            "rule": {
                "string": "Detect_PrivEscalation_via_GTFOBINS",
                "order": 3
            },
            "type": {
                "string": "Undefined",
                "order": 1
            },
            "category": {
                "string": "Undefined",
                "order": 0
            },
            "siem_inc_id": {
                "string": "**8249060a-4748-43dc-9333-6b87b979c569**", 
                "order": 2
            }
        },
        "description": "#### C\u0441\u044b\u043b\u043a\u0430 INC-15830 \u0432 TEST SIEM: */incident/incidents/view/8249060a-4748-43dc-9333-6b87b979c569?groupId=all_incidents&tabName=tasks**",
        "tags": [
            "Detect_PrivEscalation_via_GTFOBINS"
        ],
        "fromAlert": {
            "_id": "~164831232",
            "type": "Undefined",
            "source": "TEST SIEM",
            "sourceRef": "INC-15823"
        }
    },

 "objectType": "case",
    "objectId": "~206028968",
    "base": true,
    "startDate": 1623061083417,
    "rootId": "~206028968",
    "requestId": "998486afe9453169:-65c6db37:179e53a9acf:-8000:1503",
    "object": {
        "_id": "~206028968",
        "id": "~206028968",
        "createdBy": "test@mail.ru",
        "updatedBy": null,
        "createdAt": 1623060982047,
        "updatedAt": null,
        "_type": "case",
        "caseId": 78,
        "title": "\u041f\u043e\u043b\u044c\u0437\u043e\u0432\u0430\u0442\u0435\u043b\u044c root \u0441 UID 0 \u043f\u0440\u0438 \u043f\u043e\u043c\u043e\u0449\u0438 \u0443\u0442\u0438\u043b\u0438\u0442\u044b sudo \u0432\u044b\u043f\u043e\u043b\u043d\u0438\u043b \u043a\u043e\u043c\u0430\u043d\u0434\u0443 sudo bash /tmp/61a7-3f05-7851-586d \u0434\u043b\u044f \u043f\u043e\u0432\u044b\u0448\u0435\u043d\u0438\u044f \u043f\u0440\u0438\u0432\u0438\u043b\u0435\u0433\u0438\u0439 \u043d\u0430 \u0443\u0437\u043b\u0435 122.22.23.140",
        "description": /incident/incidents/view/8249060a-4748-43dc-9333-6b87b979c569?groupId=all_incidents&tabName=tasks**",
        "severity": 3,
        "startDate": 1623060982034,
        "endDate": null,
        "impactStatus": null,
        "resolutionStatus": null,
        "tags": [
            "Detect_PrivEscalation_via_GTFOBINS"
        ],
        "flag": false,
        "tlp": 3,
        "pap": 2,
        "status": "Open",
        "summary": null,
        "owner": "test@mail.ru",
        "customFields": {
            "time-incident": {
                "date": 1623060001000,
                "order": 5
            },
            "rule": {
                "string": "Detect_PrivEscalation_via_GTFOBINS",
                "order": 4
            },
            "false": {
                "string": null,
                "order": 6
            },
            "status": {
                "string": "\u0417\u0430\u043a\u0440\u044b\u0442",
                "order": 2
            },
            "type": {
                "string": "Undefined",
                "order": 1
            },
            "category": {
                "string": "Undefined",
                "order": 0
            },
            "siem_inc_id": {
                "string": "8249060a-4748-43dc-9333-6b87b979c569",
                "order": 3
            }
        },
        "stats": {},
        "permissions": []
    },
    "organisationId": "~16464",
    "organisation": "Test"

**Update alert**:

{
    "operation": "update",
    "details": {
        "caseId": "~206028968"
    },
    "objectType": "alert",
    "objectId": "~164831232",
    "base": false,
    "startDate": 1623061083410,
    "rootId": "~206028968",
    "requestId": "998486afe9453169:-65c6db37:179e53a9acf:-8000:1503",
    "object": {
        "_id": "~206028968",
        "id": "~206028968",
        "createdBy": "test@mail.ru",
        "updatedBy": null,
        "createdAt": 1623060982047,
        "updatedAt": null,
        "_type": "case",
        "caseId": 78,
        "title": "\u\u0434\u043b\u044f \u043f\u043e\u0432\u044b\u0448\u0435\u043d\u0438\u044f \u043f\u0440\u0438\u0432\u0438\u043b\u0435\u0433\u0438\u0439 \u043d\u0430 \u0443\u0437\u043b\u0435 112.23.0",
        "description": "#### C\u0441\u044b\u043b\u043c-9333-6b87b979c569?groupId=all_incidents&tabName=tasks**",
        "severity": 3,
        "startDate": 1623060982034,
        "endDate": null,
        "impactStatus": null,
        "resolutionStatus": null,
        "tags": [
            "Detect_PrivEscalation_via_GTFOBINS"
        ],
        "flag": false,
        "tlp": 3,
        "pap": 2,
        "status": "Open",
        "summary": null,
        "owner": "test@mail.ru",
        "customFields": {
            "time-incident": {
                "date": 1623060001000,
                "order": 5
            },
            "rule": {
                "string": "Detect_PrivEscalation_via_GTFOBINS",
                "order": 4
            },
            "false": {
                "string": null,
                "order": 6
            },
            "status": {
                "string": "\u0417\u0430\u043a\u0440\u044b\u0442",
                "order": 2
            },
            "type": {
                "string": "Undefined",
                "order": 1
            },
            "category": {
                "string": "Undefined",
                "order": 0
            },
            "siem_inc_id": {
                "string": "8249060a-4748-43dc-9333-6b87b979c569",
                "order": 3
            }
        },
        "stats": {},
        "permissions": []
    },
    "organisationId": "~16464",
    "organisation": "Test"
}

Merge of the next alert to the case:

{
    "operation": "update",
    "details": {
        "customFields": {
            "time-incident": {
                "date": 1623060001000,
                "order": 4
            },
            "rule": {
                "string": "Detect_PrivEscalation_via_GTFOBINS",
                "order": 3
            },
            "type": {
                "string": "Undefined",
                "order": 1
            },
            "category": {
                "string": "Undefined",
                "order": 0
            },
            "siem_inc_id": {
                "string": "**f977da46-9bef-4b52-a257-e85d9905df9a**",
                "order": 2
            }
        },
        "description": "#### C\u0441\u044b\u043b\u043a\u0430 \u0432 s/view/f977da46-9bef-4b52-a257-e85d9905df9a?groupId=all_incidents&tabName=tasks**",
        "tags": [
            "Detect_PrivEscalation_via_GTFOBINS"
        ],
        "fromAlert": {
            "_id": "~164892832",
            "type": "Undefined",
            "source": "test SIEM",
            "sourceRef": "INC-15824"
        }
    },
    "objectType": "case",
    "objectId": "~206028968",
    "base": false,
    "startDate": 1623061083409,
    "rootId": "~206028968",
    "requestId": "998486afe9453169:-65c6db37:179e53a9acf:-8000:1503",
    "object": {
        "_id": "~206028968",
        "id": "~206028968",
        "createdBy": "test@mail.ru",
        "updatedBy": null,
        "createdAt": 1623060982047,
        "updatedAt": null,
        "_type": "case",
        "caseId": 78,
        "title": "\u43b\u0435\u0433\u0438\u0439 \u043d\u0430 \u0443\u0437\u043b\u0435 152.49.21.134",
        "description": 8249060a-4748-43dc-9333-6b87b979c569?groupId=all_incidents&tabName=tasks**",
        "severity": 3,
        "startDate": 1623060982034,
        "endDate": null,
        "impactStatus": null,
        "resolutionStatus": null,
        "tags": [
            "Detect_PrivEscalation_via_GTFOBINS"
        ],
        "flag": false,
        "tlp": 3,
        "pap": 2,
        "status": "Open",
        "summary": null,
        "owner": "test@mail.ru",
        "customFields": {
            "time-incident": {
                "date": 1623060001000,
                "order": 5
            },
            "rule": {
                "string": "Detect_PrivEscalation_via_GTFOBINS",
                "order": 4
            },
            "false": {
                "string": null,
                "order": 6
            },
            "status": {
                "string": "\u0417\u0430\u043a\u0440\u044b\u0442",
                "order": 2
            },
            "type": {
                "string": "Undefined",
                "order": 1
            },
            "category": {
                "string": "Undefined",
                "order": 0
            },
            "siem_inc_id": {
                "string": "8249060a-4748-43dc-9333-6b87b979c569", (Should be f977da46-9bef-4b52-a257-e85d9905df9a !!)
                "order": 3
            }
        },
        "stats": {},
        "permissions": []
    },
    "organisationId": "~16464",
    "organisation": "Test"
}127.0.0.1 - - [07/Jun/2021 10:18:03] "POST / HTTP/1.1" 200 -
127.0.0.1 - - [07/Jun/2021 10:18:03] "POST / HTTP/1.1" 200 -

Update alert:

{
    "operation": "update",
    "details": {
        "caseId": "~206028968"
    },
    "objectType": "alert",
    "objectId": "~164892832",
    "base": false,
    "startDate": 1623061083362,
    "rootId": "~206028968",
    "requestId": "998486afe9453169:-65c6db37:179e53a9acf:-8000:1503",
    "object": {
        "_id": "~206028968",
        "id": "~206028968",
        "createdBy": "test@mail.ru",
        "updatedBy": null,
        "createdAt": 1623060982047,
        "updatedAt": null,
        "_type": "case",
        "caseId": 78,
        "title": "\u.110",
        "description": "#### Casks**",
        "severity": 3,
        "startDate": 1623060982034,
        "endDate": null,
        "impactStatus": null,
        "resolutionStatus": null,
        "tags": [
            "Detect_PrivEscalation_via_GTFOBINS"
        ],
        "flag": false,
        "tlp": 3,
        "pap": 2,
        "status": "Open",
        "summary": null,
        "owner": "test@mail.ru",
        "customFields": {
            "time-incident": {
                "date": 1623060001000,
                "order": 5
            },
            "rule": {
                "string": "Detect_PrivEscalation_via_GTFOBINS",
                "order": 4
            },
            "false": {
                "string": null,
                "order": 6
            },
            "status": {
                "string": "\u0417\u0430\u043a\u0440\u044b\u0442",
                "order": 2
            },
            "type": {
                "string": "Undefined",
                "order": 1
            },
            "category": {
                "string": "Undefined",
                "order": 0
            },
            "siem_inc_id": {
                "string": "8249060a-4748-43dc-9333-6b87b979c569",
                "order": 3
            }
        },
        "stats": {},
        "permissions": []
    },
    "organisationId": "~16464",
    "organisation": "Test"
}

Merge new alert:

{
    "operation": "update",
    "details": {
        "customFields": {
            "time-incident": {
                "date": 1623060002000,
                "order": 4
            },
            "rule": {
                "string": "Detect_PrivEscalation_via_GTFOBINS",
                "order": 3
            },
            "type": {
                "string": "Undefined",
                "order": 1
            },
            "category": {
                "string": "Undefined",
                "order": 0
            },
            "siem_inc_id": {
                "string": "**b2b8ff34-6772-43d2-9e12-c3f8b92d445a**",
                "order": 2
            }
        },
        "description": "#asks**",
        "tags": [
            "Detect_PrivEscalation_via_GTFOBINS"
        ],
        "fromAlert": {
            "_id": "~164909216",
            "type": "Undefined",
            "source": "TEST SIEM",
            "sourceRef": "INC-15825"
        }
    },
    "objectType": "case",
    "objectId": "~206028968",
    "base": false,
    "startDate": 1623061083361,
    "rootId": "~206028968",
    "requestId": "998486afe9453169:-65c6db37:179e53a9acf:-8000:1503",
    "object": {
        "_id": "~206028968",
        "id": "~206028968",
        "createdBy": "test@mail.ru",
        "updatedBy": null,
        "createdAt": 1623060982047,
        "updatedAt": null,
        "_type": "case",
        "caseId": 78,
        "title": "\u0110",
        "description": "/view/8249060a-4748-43dc-9333-6b87b979c569?groupId=all_incidents&tabName=tasks**",
        "severity": 3,
        "startDate": 1623060982034,
        "endDate": null,
        "impactStatus": null,
        "resolutionStatus": null,
        "tags": [
            "Detect_PrivEscalation_via_GTFOBINS"
        ],
        "flag": false,
        "tlp": 3,
        "pap": 2,
        "status": "Open",
        "summary": null,
        "owner": "test@mail.ru",
        "customFields": {
            "time-incident": {
                "date": 1623060001000,
                "order": 5
            },
            "rule": {
                "string": "Detect_PrivEscalation_via_GTFOBINS",
                "order": 4
            },
            "false": {
                "string": null,
                "order": 6
            },
            "status": {
                "string": "\u0417\u0430\u043a\u0440\u044b\u0442",
                "order": 2
            },
            "type": {
                "string": "Undefined",
                "order": 1
            },
            "category": {
                "string": "Undefined",
                "order": 0
            },
            "siem_inc_id": {
                "string": "**8249060a-4748-43dc-9333-6b87b979c569**", (Should be b2b8ff34-6772-43d2-9e12-c3f8b92d445a)
                "order": 3
            }
        },
        "stats": {},
        "permissions": []
    },
    "organisationId": "~16464",
    "organisation": "Test"
}127.0.0.1 - - [07/Jun/2021 10:18:03] "POST / HTTP/1.1" 200 -
127.0.0.1 - - [07/Jun/2021 10:18:03] "POST / HTTP/1.1" 200 -

Update alert:

{
    "operation": "update",
    "details": {
        "caseId": "~206028968"
    },
    "objectType": "alert",
    "objectId": "~164909216",
    "base": false,
    "startDate": 1623061083303,
    "rootId": "~206028968",
    "requestId": "998486afe9453169:-65c6db37:179e53a9acf:-8000:1503",
    "object": {
        "_id": "~206028968",
        "id": "~206028968",
        "createdBy": "test@mail.ru",
        "updatedBy": null,
        "createdAt": 1623060982047,
        "updatedAt": null,
        "_type": "case",
        "caseId": 78,
        "title": "\",
        "description": "#me=tasks**",
        "severity": 3,
        "startDate": 1623060982034,
        "endDate": null,
        "impactStatus": null,
        "resolutionStatus": null,
        "tags": [
            "Detect_PrivEscalation_via_GTFOBINS"
        ],
        "flag": false,
        "tlp": 3,
        "pap": 2,
        "status": "Open",
        "summary": null,
        "owner": "test@mail.ru",
        "customFields": {
            "time-incident": {
                "date": 1623060001000,
                "order": 5
            },
            "rule": {
                "string": "Detect_PrivEscalation_via_GTFOBINS",
                "order": 4
            },
            "false": {
                "string": null,
                "order": 6
            },
            "status": {
                "string": "\u0417\u0430\u043a\u0440\u044b\u0442",
                "order": 2
            },
            "type": {
                "string": "Undefined",
                "order": 1
            },
            "category": {
                "string": "Undefined",
                "order": 0
            },
            "siem_inc_id": {
                "string": "**8249060a-4748-43dc-9333-6b87b979c569**",
                "order": 3
            }
        },
        "stats": {},
        "permissions": []
    },
    "organisationId": "~16464",
    "organisation": "Test"
}

Merge new alert in case:

{
    "operation": "update",
    "details": {
        "customFields": {
            "time-incident": {
                "date": 1623060003000,
                "order": 4
            },
            "rule": {
                "string": "Detect_PrivEscalation_via_GTFOBINS",
                "order": 3
            },
            "type": {
                "string": "Undefined",
                "order": 1
            },
            "category": {
                "string": "Undefined",
                "order": 0
            },
            "siem_inc_id": {
                "string": "**5e285162-8436-4651-8e20-b548fa149fc3**",
                "order": 2
            }
        },
        "description": "5e285162-8436-4651-8e20-b548fa149fc3?groupId=all_incidents&tabName=tasks**",
        "tags": [
            "Detect_PrivEscalation_via_GTFOBINS"
        ],
        "fromAlert": {
            "_id": "~205938856",
            "type": "Undefined",
            "source": "TEST SIEM",
            "sourceRef": "INC-15826"
        }
    },
    "objectType": "case",
    "objectId": "~206028968",
    "base": false,
    "startDate": 1623061083300,
    "rootId": "~206028968",
    "requestId": "998486afe9453169:-65c6db37:179e53a9acf:-8000:1503",
    "object": {
        "_id": "~206028968",
        "id": "~206028968",
        "createdBy": "test@mail.ru",
        "updatedBy": null,
        "createdAt": 1623060982047,
        "updatedAt": null,
        "_type": "case",
        "caseId": 78,
        "title": "10",
        "description": "8249060a-4748-43dc-9333-6b87b979c569?groupId=all_incidents&tabName=tasks**",
        "severity": 3,
        "startDate": 1623060982034,
        "endDate": null,
        "impactStatus": null,
        "resolutionStatus": null,
        "tags": [
            "Detect_PrivEscalation_via_GTFOBINS"
        ],
        "flag": false,
        "tlp": 3,
        "pap": 2,
        "status": "Open",
        "summary": null,
        "owner": "test@mail.ru",
        "customFields": {
            "time-incident": {
                "date": 1623060001000,
                "order": 5
            },
            "rule": {
                "string": "Detect_PrivEscalation_via_GTFOBINS",
                "order": 4
            },
            "false": {
                "string": null,
                "order": 6
            },
            "status": {
                "string": "\u0417\u0430\u043a\u0440\u044b\u0442",
                "order": 2
            },
            "type": {
                "string": "Undefined",
                "order": 1
            },
            "category": {
                "string": "Undefined",
                "order": 0
            },
            "siem_inc_id": {
                "string": "**8249060a-4748-43dc-9333-6b87b979c569**", (Should be 5e285162-8436-4651-8e20-b548fa149fc3)
                "order": 3 
            }
        },
        "stats": {},
        "permissions": []
    },
    "organisationId": "~16464",
    "organisation": "Test"
}127.0.0.1 - - [07/Jun/2021 10:18:03] "POST / HTTP/1.1" 200 -
127.0.0.1 - - [07/Jun/2021 10:18:03] "POST / HTTP/1.1" 200 -

**Update case**:

{
    "operation": "update",
    "details": {
        "caseId": "~206028968"
    },
    "objectType": "alert",
    "objectId": "~205938856",
    "base": false,
    "startDate": 1623061083247,
    "rootId": "~206028968",
    "requestId": "998486afe9453169:-65c6db37:179e53a9acf:-8000:1503",
    "object": {
        "_id": "~206028968",
        "id": "~206028968",
        "createdBy": "test@mail.ru",
        "updatedBy": null,
        "createdAt": 1623060982047,
        "updatedAt": null,
        "_type": "case",
        "caseId": 78,
        "title": "\1.110",
        "description": "view/8249060a-4748-43dc-9333-6b87b979c569?groupId=all_incidents&tabName=tasks**",
        "severity": 3,
        "startDate": 1623060982034,
        "endDate": null,
        "impactStatus": null,
        "resolutionStatus": null,
        "tags": [
            "Detect_PrivEscalation_via_GTFOBINS"
        ],
        "flag": false,
        "tlp": 3,
        "pap": 2,
        "status": "Open",
        "summary": null,
        "owner": "test@mail.ru",
        "customFields": {
            "time-incident": {
                "date": 1623060001000,
                "order": 5
            },
            "rule": {
                "string": "Detect_PrivEscalation_via_GTFOBINS",
                "order": 4
            },
            "false": {
                "string": null,
                "order": 6
            },
            "status": {
                "string": "\u0417\u0430\u043a\u0440\u044b\u0442",
                "order": 2
            },
            "type": {
                "string": "Undefined",
                "order": 1
            },
            "category": {
                "string": "Undefined",
                "order": 0
            },
            "siem_inc_id": {
                "string": "8249060a-4748-43dc-9333-6b87b979c569",
                "order": 3
            }
        },
        "stats": {},
        "permissions": []
    },
    "organisationId": "~16464",
    "organisation": "Test"
}

I really hope that you understand everything, I tried to describe the problem in as much detail as possible.

nadouani commented 3 years ago

Sorry I don't really understand the issue? What's the duplicated data?

crackytsi commented 3 years ago

@Gigazo1d So you expected, that the custom-field (siem_inc_id) from the last merged alert gets set? @nadouani : I have in my mind, that all unique custom-fields gets merged and are displayed, right?

Gigazo1d commented 3 years ago

@Gigazo1d So you expected, that the custom-field (siem_inc_id) from the last merged alert gets set?

No, this is how it works now. Custom-field (siem_inc_id) from the first (not last, I was wrong) merged alert gets set.

On the contrary, I expect custom-field (siem_inc_id) to be unique, and don't it takes the value from the first merged alert.

Gigazo1d commented 3 years ago

What's the duplicated data?

Pay attention to "object" -> "customFields" -> "siem_inc_id" it is the same everywhere - (8249060a-4748-43dc-9333-6b87b979c569 - first merged alert):

hive

It should unique and equals "details" -> "customFields" -> "siem_inc_id"

screen

nadouani commented 3 years ago

This is my understanding:

TheHive here won't update the case with custom fields except if the custom field is new or is not already set on the case. (in case.customFields. siem_inc_id for instance)

You are most certainly expecting the siem_inc_id to be multi valued here in the case, right?

nadouani commented 3 years ago

The issue's title isn't relevant btw

oohoow commented 3 years ago

Hello, @nadouani! I'll try to explain the bug in other words.

Bug description: If we merge 2 or more alerts to the case at the same time, TheHive sends only webhooks about the last alert merged to the case, but not about all the alerts merged to the case (I filter the received webhook data by the following fields: "operation": "update", "objectType": "alert")

For understanding, I will explain with an example:

We have "Alert1", "Alert2", "Alert3". We merge them at the same time in the "testСase".

As a result, TheHive (version 4.1.4-1) sends only 3 identical webhooks about the "Alert3" update. Although TheHive (version 3.4.1) sent one webhook about the "Alert1" update, one webhook about the "Alert2" update, and finally one webhook about the "Alert3" update.

I believe that in version 4.1.4-1 it should work like in version 3.4.1 :)

nadouani commented 3 years ago

Thanks @oohoow this explanation is much more clear. I don't know if it's the same as the originating github issue here (are you from the same team?)

@To-om this requires a verification of /api/alert/merge/_bulk API.

Gigazo1d commented 3 years ago

@nadouani It coincides with our problem. The teams are different :)

Gigazo1d commented 3 years ago

Hello @nadouani are there results on the issue?