search

TheHive-Project / Cortex

Cortex: a Powerful Observable Analysis and Active Response Engine
https://thehive-project.org
GNU Affero General Public License v3.0
1.28k stars 218 forks source link

Initial database-preparation ("migration") of cortex 3.1.6 does not work #421

Closed crackytsi closed 1 year ago

crackytsi commented 2 years ago

Request Type

Bug

Work Environment

Question Answer
OS version (server) RedHat
OS version (client) 10
Cortex version / git hash 3.1.6
Package Type Binary
Browser type & version Chrome
Elastic-Search Version 7.17.4

Problem Description

We try to install Cortex 3.1.6 on a new system. After click of "Migration of Database" button the system is freezing and the following error appears in log-file:

2022-06-23 15:08:51,936 [WARN] from org.elastic4play.database.SearchWithScroll in application-akka.actor.default-dispatcher-18 - Search error
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of `com.sksamuel.elastic4s.requests.searches.Total` (although at least one Creator exists): no int/Int-argument constructor/factory method to deserialize from Number value (0)
 at [Source: (String)"{"_scroll_id":"DnF1ZXJ5VGhlbkZldGNoBQAAAAAAABfeFm13Qy1WU0FFU0VLclNKdVRKZGljT3cAAAAAAAAX4RZtd0MtVlNBRVNFS3JTSnVUSmRpY093AAAAAAAAF98WbXdDLVZTQUVTRUtyU0p1VEpkaWNPdwAAAAAAABfiFm13Qy1WU0FFU0VLclNKdVRKZGljT3cAAAAAAAAX4BZtd0MtVlNBRVNFS3JTSnVUSmRpY093","took":1,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}"; line: 1, column: 349] (through reference chain: com.sksamuel.elastic4s.requests.searches.SearchResponse["hits"]->com.sksamuel.elastic4s.requests.searches.SearchHits["total"])

After that user retrieval failes with error code 500. What can we do?

Possible Solutions

?

Complementary information

Excerpt from application.log:

2022-06-23 15:08:50,423 [INFO] from org.thp.cortex.services.AccessLogFilter in application-akka.actor.default-dispatcher-10 - 192.168.200.114 GET /api/status took 3ms and returned 200 278 bytes
2022-06-23 15:08:51,898 [INFO] from org.thp.cortex.services.AccessLogFilter in application-akka.actor.default-dispatcher-11 - 192.168.200.114 GET /api/status took 4ms and returned 200 278 bytes
2022-06-23 15:08:51,936 [WARN] from org.elastic4play.database.SearchWithScroll in application-akka.actor.default-dispatcher-18 - Search error
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of `com.sksamuel.elastic4s.requests.searches.Total` (although at least one Creator exists): no int/Int-argument constructor/factory method to deserialize from Number value (0)
 at [Source: (String)"{"_scroll_id":"DnF1ZXJ5VGhlbkZldGNoBQAAAAAAABfeFm13Qy1WU0FFU0VLclNKdVRKZGljT3cAAAAAAAAX4RZtd0MtVlNBRVNFS3JTSnVUSmRpY093AAAAAAAAF98WbXdDLVZTQUVTRUtyU0p1VEpkaWNPdwAAAAAAABfiFm13Qy1WU0FFU0VLclNKdVRKZGljT3cAAAAAAAAX4BZtd0MtVlNBRVNFS3JTSnVUSmRpY093","took":1,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}"; line: 1, column: 349] (through reference chain: com.sksamuel.elastic4s.requests.searches.SearchResponse["hits"]->com.sksamuel.elastic4s.requests.searches.SearchHits["total"])
        at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
        at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1728)
        at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1353)
        at com.fasterxml.jackson.databind.deser.ValueInstantiator.createFromInt(ValueInstantiator.java:324)
        at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromInt(StdValueInstantiator.java:376)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromNumber(BeanDeserializerBase.java:1442)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:198)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:186)
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:542)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:563)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:438)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1405)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:351)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:184)
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:542)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:563)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:438)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1405)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:351)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:184)
        at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4674)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3629)
        at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue(ScalaObjectMapper.scala:206)
        at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue$(ScalaObjectMapper.scala:205)
        at com.sksamuel.elastic4s.JacksonSupport$$anon$1.readValue(JacksonSupport.scala:11)
        at com.sksamuel.elastic4s.ResponseHandler$.fromEntity(ResponseHandler.scala:42)
        at com.sksamuel.elastic4s.DefaultResponseHandler.handle(ResponseHandler.scala:56)
        at com.sksamuel.elastic4s.ElasticClient.$anonfun$execute$1(ElasticClient.scala:53)
        at scala.util.Success.$anonfun$map$1(Try.scala:255)
        at scala.util.Success.map(Try.scala:213)
        at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
        at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
        at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
        at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63)
        at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:85)
        at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100)
        at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
org.elastic4play.AuthenticationError: Authentication using API key is not supported
        at org.elastic4play.services.AuthSrv.authenticate(UserSrv.scala:48)
        at org.elastic4play.services.AuthSrv.authenticate$(UserSrv.scala:47)
        at org.thp.cortex.services.LocalAuthSrv.authenticate(LocalAuthSrv.scala:15)
        at org.elastic4play.services.auth.MultiAuthSrv.$anonfun$authenticate$3(MultiAuthSrv.scala:58)
        at org.elastic4play.services.auth.MultiAuthSrv$$anonfun$$nestedInanonfun$forAllAuthProvider$1$1.applyOrElse(MultiAuthSrv.scala:43)
        at org.elastic4play.services.auth.MultiAuthSrv$$anonfun$$nestedInanonfun$forAllAuthProvider$1$1.applyOrElse(MultiAuthSrv.scala:41)
        at scala.concurrent.Future.$anonfun$recoverWith$1(Future.scala:417)
        at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
        at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63)
        at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:85)
        at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100)
        at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
com.fasterxml.jackson.databind.exc.MismatchedInputException: Cannot construct instance of `com.sksamuel.elastic4s.requests.searches.Total` (although at least one Creator exists): no int/Int-argument constructor/factory method to deserialize from Number value (0)
 at [Source: (String)"{"took":2,"timed_out":false,"_shards":{"total":5,"successful":5,"skipped":0,"failed":0},"hits":{"total":0,"max_score":null,"hits":[]}}"; line: 1, column: 105] (through reference chain: com.sksamuel.elastic4s.requests.searches.SearchResponse["hits"]->com.sksamuel.elastic4s.requests.searches.SearchHits["total"])
        at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:63)
        at com.fasterxml.jackson.databind.DeserializationContext.reportInputMismatch(DeserializationContext.java:1728)
        at com.fasterxml.jackson.databind.DeserializationContext.handleMissingInstantiator(DeserializationContext.java:1353)
        at com.fasterxml.jackson.databind.deser.ValueInstantiator.createFromInt(ValueInstantiator.java:324)
        at com.fasterxml.jackson.databind.deser.std.StdValueInstantiator.createFromInt(StdValueInstantiator.java:376)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromNumber(BeanDeserializerBase.java:1442)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeOther(BeanDeserializer.java:198)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:186)
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:542)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:563)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:438)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1405)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:351)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:184)
        at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:542)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeWithErrorWrapping(BeanDeserializer.java:563)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer._deserializeUsingPropertyBased(BeanDeserializer.java:438)
        at com.fasterxml.jackson.databind.deser.BeanDeserializerBase.deserializeFromObjectUsingNonDefault(BeanDeserializerBase.java:1405)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserializeFromObject(BeanDeserializer.java:351)
        at com.fasterxml.jackson.databind.deser.BeanDeserializer.deserialize(BeanDeserializer.java:184)
        at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:322)
        at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4674)
        at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3629)
        at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue(ScalaObjectMapper.scala:206)
        at com.fasterxml.jackson.module.scala.ScalaObjectMapper.readValue$(ScalaObjectMapper.scala:205)
        at com.sksamuel.elastic4s.JacksonSupport$$anon$1.readValue(JacksonSupport.scala:11)
        at com.sksamuel.elastic4s.ResponseHandler$.fromEntity(ResponseHandler.scala:42)
        at com.sksamuel.elastic4s.DefaultResponseHandler.handle(ResponseHandler.scala:56)
        at com.sksamuel.elastic4s.ElasticClient.$anonfun$execute$1(ElasticClient.scala:53)
        at scala.util.Success.$anonfun$map$1(Try.scala:255)
        at scala.util.Success.map(Try.scala:213)
        at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
        at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
        at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
        at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:63)
        at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:100)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:85)
        at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:100)
        at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:49)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
2022-06-23 15:08:51,959 [WARN] from org.thp.cortex.services.ErrorHandler in application-akka.actor.default-dispatcher-11 - GET /api/user/current returned 500
2022-06-23 15:08:51,960 [INFO] from org.thp.cortex.services.AccessLogFilter in application-akka.actor.default-dispatcher-11 - 192.168.200.114 GET /api/user/current took 43ms and returned 500 647 bytes

We tried to downgrade to Cortex 3.1.4. Error is then:

2022-06-23 15:25:50,343 [ERROR] from org.elastic4play.database.DBConfiguration in application-akka.actor.default-dispatcher-22 - ElasticSearch request failure: POST:/cortex_6/_update/foo?_source=true&refresh=wait_for&routing=foo&retry_on_conflict=5
StringEntity({"script":{"source":"ctx._source[\"password\"]=params.param0;ctx._source[\"updatedBy\"]=params.param1;ctx._source[\"updatedAt\"]=params.param2","params":{"param0":"捶櫨돏䫯ꬦ䢹⥬呫ᾑ䳚,f35dd473e817afbfa51b3da066a8b32ba577de10a773166bed71bb59e170ca16","param1":"init","param2":1655990749952}}},Some(application/json))
 => ElasticError(illegal_argument_exception,request [/cortex_6/_update/foo] contains unrecognized parameters: [_source], [retry_on_conflict],None,None,None,List(ElasticError(illegal_argument_exception,request [/cortex_6/_update/foo] contains unrecognized parameters: [_source], [retry_on_conflict],None,None,None,null,None,None,None,List())),None,None,None,List())
2022-06-23 15:25:50,345 [INFO] from org.thp.cortex.services.ErrorHandler in application-akka.actor.default-dispatcher-22 - POST /api/user returned 500

Error after downgrade to Cortex 3.1.1:

2022-06-23 15:32:35,862 [ERROR] from org.elastic4play.database.DBConfiguration in application-akka.actor.default-dispatcher-6 - ElasticSearch request failure: POST:/cortex_6/_update/foo?_source=true&refresh=wait_for&routing=foo&retry_on_conflict=5
StringEntity({"script":{"source":"ctx._source[\"password\"]=params.param0;ctx._source[\"updatedBy\"]=params.param1;ctx._source[\"updatedAt\"]=params.param2","params":{"param0":"툈់狈㏒ꎞ㶵ꫵ뽰얙࢔,832a06b9a4544420ec2b884dafedcdcca07795c5944d7017d0ad38a427d1be6a","param1":"init","param2":1655991155805}}},Some(application/json))
 => ElasticError(illegal_argument_exception,request [/cortex_6/_update/foo] contains unrecognized parameters: [_source], [retry_on_conflict],None,None,None,List(ElasticError(illegal_argument_exception,request [/cortex_6/_update/foo] contains unrecognized parameters: [_source], [retry_on_conflict],None,None,None,null,None,None,None,List())),None,None,None,List())
2022-06-23 15:32:35,865 [INFO] from org.thp.cortex.services.ErrorHandler in application-akka.actor.default-dispatcher-6 - POST /api/user returned 500
org.elastic4play.InternalError: Unknown error: ElasticError(illegal_argument_exception,request [/cortex_6/_update/foo] contains unrecognized parameters: [_source], [retry_on_conflict],None,None,None,List(ElasticError(illegal_argument_exception,request [/cortex_6/_update/foo] contains unrecognized parameters: [_source], [retry_on_conflict],None,None,None,null,None,None,None,List())),None,None,None,List())
        at org.elastic4play.database.DBConfiguration.$anonfun$execute$2(DBConfiguration.scala:158)
        at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
        at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
        at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
        at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:56)
        at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:93)
        at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
        at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:85)
        at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:93)
        at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:48)
        at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
        at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
        at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
        at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
        at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
To-om commented 2 years ago

I can't reproduce the issue. Can you share your Cortex configuration and also the configuration of Elasticsearch ?

crackytsi commented 1 year ago

We installed a complete new system, working. Very confusing...