TheHive-Project / CortexDocs

Documentation of Cortex
https://docs.strangebee.com/cortex
GNU Affero General Public License v3.0
170 stars 122 forks source link

Create Cortex administrator account #86

Closed asgharali1 closed 2 years ago

asgharali1 commented 2 years ago

I am trying to create Cortex admin user, and I am receiving the following exception. Please advice.

2022-02-04 12:01:25,424 [WARN] from org.elastic4play.database.SearchWithScroll in application-akka.actor.default-dispatcher-6 - Search error
com.sksamuel.elastic4s.http.JavaClientExceptionWrapper: java.net.ConnectException: Connection refused
    at com.sksamuel.elastic4s.http.JavaClient$$anon$1.onFailure(JavaClient.scala:69)
    at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onDefinitiveFailure(RestClient.java:617)
    at org.elasticsearch.client.RestClient$1.failed(RestClient.java:375)
    at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:137)
    at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.executionFailed(DefaultClientExchangeHandlerImpl.java:101)
    at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.failed(AbstractClientExchangeHandler.java:426)
    at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.connectionRequestFailed(AbstractClientExchangeHandler.java:348)
    at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.access$100(AbstractClientExchangeHandler.java:62)
    at org.apache.http.impl.nio.client.AbstractClientExchangeHandler$1.failed(AbstractClientExchangeHandler.java:392)
    at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:137)
    at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$1.failed(PoolingNHttpClientConnectionManager.java:316)
    at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:137)
    at org.apache.http.nio.pool.RouteSpecificPool.failed(RouteSpecificPool.java:162)
    at org.apache.http.nio.pool.AbstractNIOConnPool.requestFailed(AbstractNIOConnPool.java:609)
    at org.apache.http.nio.pool.AbstractNIOConnPool$InternalSessionRequestCallback.failed(AbstractNIOConnPool.java:889)
    at org.apache.http.impl.nio.reactor.SessionRequestImpl.failed(SessionRequestImpl.java:162)
    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:176)
    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148)
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
    at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
    at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.net.ConnectException: Connection refused
    at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174)
    ... 5 common frames omitted
2022-02-04 12:01:25,432 [WARN] from org.elastic4play.database.SearchWithScroll in application-akka.actor.default-dispatcher-10 - Search error
com.sksamuel.elastic4s.http.JavaClientExceptionWrapper: java.net.ConnectException: Connection refused
    at com.sksamuel.elastic4s.http.JavaClient$$anon$1.onFailure(JavaClient.scala:69)
    at org.elasticsearch.client.RestClient$FailureTrackingResponseListener.onDefinitiveFailure(RestClient.java:617)
    at org.elasticsearch.client.RestClient$1.failed(RestClient.java:375)
    at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:137)
    at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.executionFailed(DefaultClientExchangeHandlerImpl.java:101)
    at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.failed(AbstractClientExchangeHandler.java:426)
    at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.connectionRequestFailed(AbstractClientExchangeHandler.java:348)
    at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.access$100(AbstractClientExchangeHandler.java:62)
    at org.apache.http.impl.nio.client.AbstractClientExchangeHandler$1.failed(AbstractClientExchangeHandler.java:392)
    at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:137)
    at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager$1.failed(PoolingNHttpClientConnectionManager.java:316)
    at org.apache.http.concurrent.BasicFuture.failed(BasicFuture.java:137)
    at org.apache.http.nio.pool.RouteSpecificPool.failed(RouteSpecificPool.java:162)
    at org.apache.http.nio.pool.AbstractNIOConnPool.requestFailed(AbstractNIOConnPool.java:609)
    at org.apache.http.nio.pool.AbstractNIOConnPool$InternalSessionRequestCallback.failed(AbstractNIOConnPool.java:889)
    at org.apache.http.impl.nio.reactor.SessionRequestImpl.failed(SessionRequestImpl.java:162)
    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:176)
    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvents(DefaultConnectingIOReactor.java:148)
    at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor.execute(AbstractMultiworkerIOReactor.java:351)
    at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.execute(PoolingNHttpClientConnectionManager.java:221)
    at org.apache.http.impl.nio.client.CloseableHttpAsyncClientBase$1.run(CloseableHttpAsyncClientBase.java:64)
    at java.base/java.lang.Thread.run(Thread.java:829)
Caused by: java.net.ConnectException: Connection refused
    at java.base/sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
    at java.base/sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:777)
    at org.apache.http.impl.nio.reactor.DefaultConnectingIOReactor.processEvent(DefaultConnectingIOReactor.java:174)
    ... 5 common frames omitted
2022-02-04 12:01:25,482 [INFO] from play.api.Play in main - Application started (Prod) (no global state)
2022-02-04 12:01:26,079 [INFO] from play.core.server.AkkaHttpServer in main - Enabling HTTP/2 on Akka HTTP server...
2022-02-04 12:01:26,080 [INFO] from play.core.server.AkkaHttpServer in main - Listening for HTTP on /0:0:0:0:0:0:0:0:9001
2022-02-04 14:19:12,274 [ERROR] from org.elastic4play.controllers.Authenticated in application-akka.actor.default-dispatcher-14 - Authentication failure:
    session: AuthenticationError User session not found
    pki: AuthenticationError Certificate authentication is not configured
    key: AuthenticationError Authentication header not found
    init: AuthenticationError Use of initial user is forbidden because users exist in database
2022-02-04 14:19:12,277 [INFO] from org.thp.cortex.services.ErrorHandler in application-akka.actor.default-dispatcher-14 - POST /api/stream returned 401
org.elastic4play.AuthenticationError: Authentication failure
    at org.elastic4play.controllers.Authenticated.$anonfun$getContext$4(Authenticated.scala:272)
    at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
    at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:56)
    at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:93)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:85)
    at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:93)
    at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:48)
    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
2022-02-04 14:21:37,382 [ERROR] from org.elastic4play.database.DBConfiguration in application-akka.actor.default-dispatcher-23 - ElasticSearch request failure: POST:/cortex_6/_search?
StringEntity({"query":{"match":{"relations":{"query":"user"}}},"size":0},Some(application/json))
 => ElasticError(index_not_found_exception,no such index,Some(_na_),Some(cortex_6),None,List(ElasticError(index_not_found_exception,no such index,Some(_na_),Some(cortex_6),None,null,None,None,None,List())),None,None,None,List())
2022-02-04 14:21:37,436 [ERROR] from org.elastic4play.database.DBConfiguration in application-akka.actor.default-dispatcher-14 - ElasticSearch request failure: POST:/cortex_6/_search?
StringEntity({"query":{"match":{"relations":{"query":"user"}}},"size":0},Some(application/json))
 => ElasticError(index_not_found_exception,no such index,Some(_na_),Some(cortex_6),None,List(ElasticError(index_not_found_exception,no such index,Some(_na_),Some(cortex_6),None,null,None,None,None,List())),None,None,None,List())
2022-02-04 14:21:38,707 [INFO] from com.sksamuel.elastic4s.http.JavaClient$ in application-akka.actor.default-dispatcher-21 - Creating HTTP client on http://127.0.0.1:9200
2022-02-04 14:21:38,752 [INFO] from com.sksamuel.elastic4s.http.JavaClient$ in application-akka.actor.default-dispatcher-21 - Creating HTTP client on http://127.0.0.1:9200
2022-02-04 14:21:38,770 [INFO] from com.sksamuel.elastic4s.http.JavaClient$ in application-akka.actor.default-dispatcher-21 - Creating HTTP client on http://127.0.0.1:9200
2022-02-04 14:21:38,787 [INFO] from com.sksamuel.elastic4s.http.JavaClient$ in application-akka.actor.default-dispatcher-21 - Creating HTTP client on http://127.0.0.1:9200
2022-02-04 14:21:38,829 [INFO] from com.sksamuel.elastic4s.http.JavaClient$ in application-akka.actor.default-dispatcher-21 - Creating HTTP client on http://127.0.0.1:9200
2022-02-04 14:21:38,846 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Create a new empty database
2022-02-04 14:21:38,847 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrate database from version 0, add operations for version 2
2022-02-04 14:21:38,851 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrate database from version 0, add operations for version 3
2022-02-04 14:21:38,852 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrate database from version 0, add operations for version 4
2022-02-04 14:21:38,852 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrate database from version 0, add operations for version 5
2022-02-04 14:21:38,852 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrate database from version 0, add operations for version 6
2022-02-04 14:21:39,248 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrating 0 entities from sequence
2022-02-04 14:21:39,277 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrating 0 entities from artifact
2022-02-04 14:21:39,279 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrating 0 entities from audit
2022-02-04 14:21:39,281 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrating 0 entities from data
2022-02-04 14:21:39,283 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrating 0 entities from dblist
2022-02-04 14:21:39,284 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrating 0 entities from job
2022-02-04 14:21:39,285 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrating 0 entities from organization
2022-02-04 14:21:39,286 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrating 0 entities from report
2022-02-04 14:21:39,288 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrating 0 entities from user
2022-02-04 14:21:39,289 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrating 0 entities from worker
2022-02-04 14:21:39,292 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-21 - Migrating 0 entities from workerConfig
2022-02-04 14:21:40,109 [INFO] from org.elastic4play.services.MigrationSrv in application-akka.actor.default-dispatcher-23 - End of migration
2022-02-04 14:30:07,229 [ERROR] from org.elastic4play.database.DBConfiguration in application-akka.actor.default-dispatcher-14 - ElasticSearch request failure: POST:/cortex_6/_update/admin?_source=true&refresh=wait_for&routing=admin&retry_on_conflict=5
StringEntity({"script":{"source":"ctx._source[\"password\"]=params.param0;ctx._source[\"updatedBy\"]=params.param1;ctx._source[\"updatedAt\"]=params.param2","params":{"param0":"ڭ៥僣龚픥誚Ꞟ퓻ƺ橒,64a6a4f9446b9c0ab685af5985e6c7f6673eae2b5a5791b6cfc438fca727df94","param1":"init","param2":1643985007202}}},Some(application/json))
 => ElasticError(illegal_argument_exception,request [/cortex_6/_update/admin] contains unrecognized parameters: [_source], [retry_on_conflict],None,None,None,List(ElasticError(illegal_argument_exception,request [/cortex_6/_update/admin] contains unrecognized parameters: [_source], [retry_on_conflict],None,None,None,null,None,None,None,List())),None,None,None,List())
2022-02-04 14:30:07,230 [INFO] from org.thp.cortex.services.ErrorHandler in application-akka.actor.default-dispatcher-14 - POST /api/user returned 500
org.elastic4play.InternalError: Unknown error: ElasticError(illegal_argument_exception,request [/cortex_6/_update/admin] contains unrecognized parameters: [_source], [retry_on_conflict],None,None,None,List(ElasticError(illegal_argument_exception,request [/cortex_6/_update/admin] contains unrecognized parameters: [_source], [retry_on_conflict],None,None,None,null,None,None,None,List())),None,None,None,List())
    at org.elastic4play.database.DBConfiguration.$anonfun$execute$2(DBConfiguration.scala:158)
    at scala.concurrent.Future.$anonfun$flatMap$1(Future.scala:307)
    at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:56)
    at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:93)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:85)
    at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:93)
    at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:48)
    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
2022-02-04 14:30:07,779 [INFO] from org.thp.cortex.services.ErrorHandler in application-akka.actor.default-dispatcher-20 - GET /api/stream/rqjqbnfJUb returned 401
org.elastic4play.AuthenticationError: Authentication header not found
    at org.elastic4play.controllers.Authenticated.$anonfun$getFromApiKey$1(Authenticated.scala:143)
    at scala.Option.fold(Option.scala:251)
    at org.elastic4play.controllers.Authenticated.getFromApiKey(Authenticated.scala:143)
    at org.thp.cortex.controllers.StreamCtrl$$anonfun$1.applyOrElse(StreamCtrl.scala:101)
    at org.thp.cortex.controllers.StreamCtrl$$anonfun$1.applyOrElse(StreamCtrl.scala:101)
    at scala.concurrent.Future.$anonfun$recoverWith$1(Future.scala:417)
    at scala.concurrent.impl.Promise.$anonfun$transformWith$1(Promise.scala:41)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
    at akka.dispatch.BatchingExecutor$AbstractBatch.processBatch(BatchingExecutor.scala:56)
    at akka.dispatch.BatchingExecutor$BlockableBatch.$anonfun$run$1(BatchingExecutor.scala:93)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    at scala.concurrent.BlockContext$.withBlockContext(BlockContext.scala:85)
    at akka.dispatch.BatchingExecutor$BlockableBatch.run(BatchingExecutor.scala:93)
    at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:48)
    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:290)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1020)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1656)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1594)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:183)
asgharali1 commented 2 years ago

Ubuntu: Ubuntu 20.04.3 LTS – m5.xlarge – volume size 50GB Cortex: 3.1.4-1 Git Version: 2.25.1 Elasticsearch: 7.17.0

Elasticsearch Directory: /etc/elasticsearch/elasticsearch.yml

    cluster.name: <Name of the cluster>
    node.name: <IP>
    path.data: /var/lib/elasticsearch
    path.logs: /var/log/elasticsearch
    network.host: 0.0.0.0
    http.port: 9200
    cluster.initial_master_nodes:
        - <IP>

sudo /bin/systemctl daemon-reload
sudo systemctl enable elasticsearch.service
sudo systemctl start elasticsearch.service
sudo systemctl status elasticsearch.service
service elasticsearch restart
ps -ef|grep elasticsearch

Cortex configuration file Directory: cd /etc/cortex

Generate play.http.secret.key CLI Command: $(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w 64 | head -n 1)

vim /etc/cortex/application.conf play.http.secret.key="<Input Generated Key here>"

ElasticSearch index = cortex uri = http://127.0.0.1:9200/

ANALYZERS urls = [ "https://download.thehive-project.org/analyzers.json" "/opt/Cortex-Analyzers/analyzers" ]

RESPONDERS

  # responder location (same format as analyzer.urls)

urls = [ "https://download.thehive-project.org/responders.json" "/opt/Cortex-Analyzers/responders" ]

sudo service cortex stop
sudo service cortex status
sudo service cortex start