search

TheHive-Project / TheHive

TheHive: a Scalable, Open Source and Free Security Incident Response Platform
https://thehive-project.org
GNU Affero General Public License v3.0
3.39k stars 618 forks source link

[Bug] MISP export controller, way to handle export iocs #2222

Open torsolaso opened 2 years ago

torsolaso commented 2 years ago

Request Type

Bug

Work Environment

Question Answer
OS version (server) Ubunt
TheHive version / git hash 4.1.7
Database Cassandra
Index type Elasticsearch

Problem Description

When malware samples (datatype file ) are exported from a case to MISP, a congestion situation occurs in MISP in which an attempt is made to upload several samples at the same time and gives an error.

This causes a link between the case and MISP not to be assigned, causing each export to be a new, different looping event.

As a consequence, it is not possible to successfully export the case

Steps to Reproduce

  1. Load observables: ips, domains, urls...
  2. Load 5 or more samples
  3. Click on export to MISP

Possible Solutions

Create the link between the TH case and the MISP event even if there is a failure, and detect which observable is missing to synchronize

Complementary information

Thv4 error log

2021-10-20 13:42:02,449 [WARN] from org.thp.scalligraph.ErrorHandler in application-akka.actor.default-dispatcher-18 [|] POST /api/connector/misp/export/~3134115840/MISP returned 500
2021-10-20 13:42:02,450 [INFO] from org.thp.scalligraph.AccessLogFilter in application-akka.actor.default-dispatcher-18 [0000009a|] 169.254.132.1 POST /api/connector/misp/export/~3134115840/MISP took 126071ms and returned 500 136 bytes

MISP error log

`2021-10-20 13:46:13 Error: [PDOException] SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction Request URL: /events/upload_sample/20144 Stack Trace:

0 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php(502): PDOStatement->execute()

1 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php(468): DboSource->_execute()

2 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/Database/Mysql.php(431): DboSource->execute()

3 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(2672): Mysql->update()

4 /var/www/MISP/app/Model/Attribute.php(389): Model->updateAll()

5 /var/www/MISP/app/Model/Attribute.php(485): Attribute->__alterAttributeCount()

6 /var/www/MISP/app/Lib/cakephp/lib/Cake/Event/CakeEventManager.php(242): Attribute->afterSave()

7 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(1970): CakeEventManager->dispatch()

8 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(1760): Model->_doSave()

9 /var/www/MISP/app/Model/Attribute.php(3659): Model->save()

10 /var/www/MISP/app/Model/MispObject.php(994): Attribute->captureAttribute()

11 /var/www/MISP/app/Controller/EventsController.php(4381): MispObject->captureObject()

12 [internal function]: EventsController->upload_sample()

13 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Controller.php(499): ReflectionMethod->invokeArgs()

14 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(193): Controller->invokeAction()

15 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(167): Dispatcher->_invoke()

16 /var/www/MISP/app/webroot/index.php(92): Dispatcher->dispatch()

17 {main}

2021-10-20 13:46:13 Error: [PDOException] SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction Request URL: /events/upload_sample/20144 Stack Trace:

0 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php(502): PDOStatement->execute()

1 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/DboSource.php(468): DboSource->_execute()

2 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Datasource/Database/Mysql.php(431): DboSource->execute()

3 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(2672): Mysql->update()

4 /var/www/MISP/app/Model/Attribute.php(389): Model->updateAll()

5 /var/www/MISP/app/Model/Attribute.php(485): Attribute->__alterAttributeCount()

6 /var/www/MISP/app/Lib/cakephp/lib/Cake/Event/CakeEventManager.php(242): Attribute->afterSave()

7 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(1970): CakeEventManager->dispatch()

8 /var/www/MISP/app/Lib/cakephp/lib/Cake/Model/Model.php(1760): Model->_doSave()

9 /var/www/MISP/app/Model/Attribute.php(3659): Model->save()

10 /var/www/MISP/app/Model/MispObject.php(994): Attribute->captureAttribute()

11 /var/www/MISP/app/Controller/EventsController.php(4381): MispObject->captureObject()

12 [internal function]: EventsController->upload_sample()

13 /var/www/MISP/app/Lib/cakephp/lib/Cake/Controller/Controller.php(499): ReflectionMethod->invokeArgs()

14 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(193): Controller->invokeAction()

15 /var/www/MISP/app/Lib/cakephp/lib/Cake/Routing/Dispatcher.php(167): Dispatcher->_invoke()

16 /var/www/MISP/app/webroot/index.php(92): Dispatcher->dispatch()

17 {main}`

MISP issue. https://github.com/MISP/MISP/issues/5004

mamoedo commented 2 years ago

@torsolaso did you manage to fix this error? I'm still getting it