search

TheHive-Project / Cortex

Cortex: a Powerful Observable Analysis and Active Response Engine
https://strangebee.com/cortex/
GNU Affero General Public License v3.0
1.33k stars 228 forks source link

Analyzer config/mismatch makes TheHive unable to use Cortex after upgrading to 3.0.1 #259

Open pettai opened 4 years ago

pettai commented 4 years ago

Request Type

Bug?

Work Environment

Question Answer
OS version (server) Ubuntu 18.04
Cortex version / git hash 3.0.1
Package Type Deb Binary
Browser type & version Firefox

Problem Description

After upgrading Cortex from 3.0.0-1 -> 3.0.1-1, logging into the UI doesn't work

Steps to Reproduce

Authentication with the right password yields this error message in the logfile:

2020-03-21 21:42:56,487 [ERROR] from org.elastic4play.controllers.Authenticated in application-akka.actor.default-dispatcher-14 - Authentication failure:
    session: AuthenticationError User session not found
    pki: AuthenticationError Certificate authentication is not configured
    key: AuthenticationError Authentication header not found
    init: AuthenticationError Use of initial user is forbidden because users exist in database
2020-03-21 21:42:56,488 [INFO] from org.thp.cortex.services.ErrorHandler in application-akka.actor.default-dispatcher-14 - GET /api/user/current returned 401
org.elastic4play.AuthenticationError: Authentication failure
    at org.elastic4play.controllers.Authenticated.$anonfun$getContext$4(Authenticated.scala:272)
[...]

Complementary information

If I try the wrong password, the log message a looks different (and the UI also gets a red popup box that says "Authentication Failure" :

2020-03-21 21:50:31,551 [INFO] from org.thp.cortex.services.ErrorHandler in application-akka.actor.default-dispatcher-14 - POST /api/login returned 401
org.elastic4play.AuthenticationError: Authentication failure
    at org.elastic4play.services.auth.MultiAuthSrv$$anonfun$authenticate$2.applyOrElse(MultiAuthSrv.scala:54)
[...]

Other then that, I see very little of log messages that hints about something being wrong... I noted that after restart, there are some complaints, but I doubt any of those are the root cause of this problem:


2020-03-21 21:12:10,489 [WARN] from application in main - /etc/cortex/application.conf: 116: analyzer.path is deprecated, use analyzer.urls instead
2020-03-21 21:12:10,489 [WARN] from application in main - /etc/cortex/application.conf: 134: responder.path is deprecated, use responder.urls instead
2020-03-21 21:12:11,974 [INFO] from org.apache.http.impl.execchain.RetryExec in jersey-client-async-executor-0 - I/O exception (java.io.IOException) caught when processing request to {}->unix://localhost:80: No such file or directory
2020-03-21 21:12:11,974 [INFO] from org.apache.http.impl.execchain.RetryExec in jersey-client-async-executor-0 - Retrying request to {}->unix://localhost:80
2020-03-21 21:12:11,975 [INFO] from org.apache.http.impl.execchain.RetryExec in jersey-client-async-executor-0 - I/O exception (java.io.IOException) caught when processing request to {}->unix://localhost:80: No such file or directory
2020-03-21 21:12:11,975 [INFO] from org.apache.http.impl.execchain.RetryExec in jersey-client-async-executor-0 - Retrying request to {}->unix://localhost:80
2020-03-21 21:12:11,975 [INFO] from org.apache.http.impl.execchain.RetryExec in jersey-client-async-executor-0 - I/O exception (java.io.IOException) caught when processing request to {}->unix://localhost:80: No such file or directory
2020-03-21 21:12:11,976 [INFO] from org.apache.http.impl.execchain.RetryExec in jersey-client-async-executor-0 - Retrying request to {}->unix://localhost:80
2020-03-21 21:12:11,980 [INFO] from org.thp.cortex.services.DockerJobRunnerSrv in main - Docker is not available
com.spotify.docker.client.exceptions.DockerException: java.util.concurrent.ExecutionException: javax.ws.rs.ProcessingException: java.io.IOException: No such file or directory ```
pettai commented 4 years ago

This seems to be an issue of an faulty analyzer. Once removed, login now works again?!? Yet, the application.log doesn't reveal any information that at least I found was useful for finding this...

pettai commented 4 years ago

Also, TheHive is acting weird because of this state. TheHive <-> Cortex integration symbol is Green in TheHive , but I can't use the Cortex server (I can't select then I want to run an Analyser)

Screen Shot 2020-03-23 at 21 18 11

How do I recover from this state? Removing the faulty Analyzer and restarting the application doesn't get it to a working state.

TheHive complaints about Cortex:

2020-03-23 20:30:44,671 [ERROR] from connectors.cortex.services.CortexAnalyzerSrv in application-akka.actor.default-dispatcher-16 - Request to Cortex fails
play.api.libs.json.JsResultException: JsResultException(errors:List(((1),List(JsonValidationError(List('dataTypeList' is undefined on object: {"_routing":"SUNET","updatedBy":"init","configuration":{"url":"http://mygrrinstanceentrypointhere:8000","username":"<USER>","password":"<PASSWORD>","auto_extract_artifacts":false,"jobCache":10,"proxy_http":null,"proxy_https":null,"cacerts":null,"jobTimeout":30,"check_tlp":true,"max_tlp":2,"check_pap":true,"max_pap":2},"analyzerDefinitionId":"GRR_1_0","author":"<AUTHOR>, SUNET","_type":"worker","description":"Search GRR for the host agent.","type":"analyzer","version":"1.0","jobCache":10,"url":"https://github.com/TheHive-Project/Cortex-Analyzers","command":"/opt/Cortex-Analyzers/analyzers/GRR/grrclient.py","license":"AGPL-V3","createdAt":1575552350981,"_parent":"SUNET","jobTimeout":30,"createdBy":"sunet-api","name":"GRR_1_0","workerDefinitionId":"GRR_1_0","_id":"b9e50829dc32c1ff1e9a4d5b103296e0","id":"b9e50829dc32c1ff1e9a4d5b103296e0","_version":10,"updatedAt":1584995288196,"baseConfig":"GRR"}),WrappedArray())))))
    at play.api.libs.json.JsReadable.$anonfun$as$2(JsReadable.scala:25)
    at play.api.libs.json.JsError.fold(JsResult.scala:56)
    at play.api.libs.json.JsReadable.as(JsReadable.scala:24)
    at play.api.libs.json.JsReadable.as$(JsReadable.scala:23)
    at play.api.libs.json.JsArray.as(JsValue.scala:91)
    at connectors.cortex.services.CortexClient.$anonfun$listAnalyzer$2(CortexClient.scala:137)
    at connectors.cortex.services.CortexClient.$anonfun$request$3(CortexClient.scala:99)
    at scala.util.Success.$anonfun$map$1(Try.scala:251)
    at scala.util.Success.map(Try.scala:209)
    at scala.concurrent.Future.$anonfun$map$1(Future.scala:288)
    at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:29)
    at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:29)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:60)
    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:55)
    at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:91)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
    at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:81)
    at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:91)
    at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:40)
    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:44)
    at akka.dispatch.forkjoin.ForkJoinTask.doExec(ForkJoinTask.java:260)
    at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1339)
    at akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979)
    at akka.dispatch.forkjoin.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107)
pettai commented 4 years ago

I don't know if this should be regarded as an Cortex issue, an TheHive issure, or problems at both ends...

nadouani commented 4 years ago

'dataTypeList' is undefined on object looks like you analyzer is missing the dataTypeList property