search

TheHive-Project / TheHive

TheHive: a Scalable, Open Source and Free Security Incident Response Platform
https://thehive-project.org
GNU Affero General Public License v3.0
3.28k stars 609 forks source link

Can't start after upgrade thehive4 (4.1.16-1) over (4.0.0-1) [Bug] #2308

Closed huesitosloco closed 2 years ago

huesitosloco commented 2 years ago

Request Type

Can't start The Hive after an upgrade [ (4.1.16-1) over (4.0.0-1)],

Work Environment

Question Answer
OS version (server) Ubuntu 18.04.4 LTS
Virtualized Env. True
Dedicated RAM 8 GB
vCPU 4
TheHive version / git hash (4.1.16-1) upgrading from (4.0.0-1)
Package Type DEB
Database Cassandra
Index type Lucene
Attachments storage Local

Problem Description

on the logs, I noticed a warning regarding a GLOBAL_OFFLINE variable that can not be overriden by the information found in /etc/thehive/application.conf

Full message: 2021-12-30 23:15:07,136 [WARN] from org.janusgraph.diskstorage.configuration.builder.ReadConfigurationBuilder in application-akka.actor.default-dispatcher-11 [|] Local setting index.search.index-name=scalligraph (Type: GLOBAL_OFFLINE) is overridden by globally managed value (janusgraph). Use the ManagementSystem interface instead of the local configuration to control this setting.

After this message, it continues to load, but encounters another problem:

2021-12-30 23:15:11,849 [TRACE] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-11 [|mgmt-33c851d7] mgmt.makePropertyKey(colour).dataType(String.class).cardinality(SINGLE).make()
2021-12-30 23:15:11,857 [ERROR] from org.thp.scalligraph.models.Database in application-akka.actor.default-dispatcher-11 [|mgmt-33c851d7] Unable to add property colour
org.thp.scalligraph.InternalError: Property colour exists with incompatible type: SINGLE:class java.lang.String Vs SINGLE:class java.lang.Integer
    at org.thp.scalligraph.janus.JanusDatabase.addProperty(JanusDatabase.scala:409)
    at org.thp.scalligraph.janus.JanusDatabase.$anonfun$addProperties$4(JanusDatabase.scala:367)
    at org.thp.scalligraph.package$RichSeq.$anonfun$toTry$3(package.scala:17)
    at scala.collection.TraversableOnce$folder$1.apply(TraversableOnce.scala:196)
    at scala.collection.TraversableOnce$folder$1.apply(TraversableOnce.scala:194)
    at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:399)
    at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:725)
    at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:725)
    at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:199)
    at scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:192)
    at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:108)
    at org.thp.scalligraph.package$RichSeq.toTry(package.scala:16)
    at org.thp.scalligraph.janus.JanusDatabase.addProperties(JanusDatabase.scala:358)
    at org.thp.scalligraph.janus.JanusDatabase.$anonfun$createSchema$4(JanusDatabase.scala:310)
    at scala.util.Success.flatMap(Try.scala:251)
    at org.thp.scalligraph.janus.JanusDatabase.$anonfun$createSchema$3(JanusDatabase.scala:309)
    at scala.util.Success.flatMap(Try.scala:251)
    at org.thp.scalligraph.janus.JanusDatabase.$anonfun$createSchema$1(JanusDatabase.scala:308)
    at org.thp.scalligraph.janus.JanusDatabase.$anonfun$managementTransaction$6(JanusDatabase.scala:292)
    at scala.util.Try$.apply(Try.scala:213)
    at org.thp.scalligraph.janus.JanusDatabase.$anonfun$managementTransaction$4(JanusDatabase.scala:292)
    at scala.util.Try$.apply(Try.scala:213)
    at org.thp.scalligraph.utils.Retry.org$thp$scalligraph$utils$Retry$$runTry(Retry.scala:61)
    at org.thp.scalligraph.utils.Retry.withTry(Retry.scala:26)
    at org.thp.scalligraph.janus.JanusDatabase.managementTransaction(JanusDatabase.scala:288)
    at org.thp.scalligraph.janus.JanusDatabase.createSchema(JanusDatabase.scala:305)
    at org.thp.scalligraph.janus.JanusDatabaseProvider.$anonfun$get$3(JanusDatabaseProvider.scala:138)
    at scala.util.Success.$anonfun$map$1(Try.scala:255)
    at scala.util.Success.map(Try.scala:213)
    at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
    at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
    at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
    at org.thp.scalligraph.ContextPropagatingDispatcher$$anon$1.$anonfun$execute$2(ContextPropagatingDisptacher.scala:57)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    at org.thp.scalligraph.DiagnosticContext$$anon$2.withContext(ContextPropagatingDisptacher.scala:77)
    at org.thp.scalligraph.ContextPropagatingDispatcher$$anon$1.$anonfun$execute$1(ContextPropagatingDisptacher.scala:57)
    at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:48)
    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)
2021-12-30 23:15:11,861 [ERROR] from org.thp.scalligraph.utils.Retry in application-akka.actor.default-dispatcher-11 [|] uncaught error, not retrying
org.thp.scalligraph.InternalError: Property colour exists with incompatible type: SINGLE:class java.lang.String Vs SINGLE:class java.lang.Integer
    at org.thp.scalligraph.janus.JanusDatabase.addProperty(JanusDatabase.scala:409)
    at org.thp.scalligraph.janus.JanusDatabase.$anonfun$addProperties$4(JanusDatabase.scala:367)
    at org.thp.scalligraph.package$RichSeq.$anonfun$toTry$3(package.scala:17)
    at scala.collection.TraversableOnce$folder$1.apply(TraversableOnce.scala:196)
    at scala.collection.TraversableOnce$folder$1.apply(TraversableOnce.scala:194)
    at scala.collection.immutable.HashMap$HashMap1.foreach(HashMap.scala:399)
    at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:725)
    at scala.collection.immutable.HashMap$HashTrieMap.foreach(HashMap.scala:725)
    at scala.collection.TraversableOnce.foldLeft(TraversableOnce.scala:199)
    at scala.collection.TraversableOnce.foldLeft$(TraversableOnce.scala:192)
    at scala.collection.AbstractTraversable.foldLeft(Traversable.scala:108)
    at org.thp.scalligraph.package$RichSeq.toTry(package.scala:16)
    at org.thp.scalligraph.janus.JanusDatabase.addProperties(JanusDatabase.scala:358)
    at org.thp.scalligraph.janus.JanusDatabase.$anonfun$createSchema$4(JanusDatabase.scala:310)
    at scala.util.Success.flatMap(Try.scala:251)
    at org.thp.scalligraph.janus.JanusDatabase.$anonfun$createSchema$3(JanusDatabase.scala:309)
    at scala.util.Success.flatMap(Try.scala:251)
    at org.thp.scalligraph.janus.JanusDatabase.$anonfun$createSchema$1(JanusDatabase.scala:308)
    at org.thp.scalligraph.janus.JanusDatabase.$anonfun$managementTransaction$6(JanusDatabase.scala:292)
    at scala.util.Try$.apply(Try.scala:213)
    at org.thp.scalligraph.janus.JanusDatabase.$anonfun$managementTransaction$4(JanusDatabase.scala:292)
    at scala.util.Try$.apply(Try.scala:213)
    at org.thp.scalligraph.utils.Retry.org$thp$scalligraph$utils$Retry$$runTry(Retry.scala:61)
    at org.thp.scalligraph.utils.Retry.withTry(Retry.scala:26)
    at org.thp.scalligraph.janus.JanusDatabase.managementTransaction(JanusDatabase.scala:288)
    at org.thp.scalligraph.janus.JanusDatabase.createSchema(JanusDatabase.scala:305)
    at org.thp.scalligraph.janus.JanusDatabaseProvider.$anonfun$get$3(JanusDatabaseProvider.scala:138)
    at scala.util.Success.$anonfun$map$1(Try.scala:255)
    at scala.util.Success.map(Try.scala:213)
    at scala.concurrent.Future.$anonfun$map$1(Future.scala:292)
    at scala.concurrent.impl.Promise.liftedTree1$1(Promise.scala:33)
    at scala.concurrent.impl.Promise.$anonfun$transform$1(Promise.scala:33)
    at scala.concurrent.impl.CallbackRunnable.run(Promise.scala:64)
    at org.thp.scalligraph.ContextPropagatingDispatcher$$anon$1.$anonfun$execute$2(ContextPropagatingDisptacher.scala:57)
    at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    at org.thp.scalligraph.DiagnosticContext$$anon$2.withContext(ContextPropagatingDisptacher.scala:77)
    at org.thp.scalligraph.ContextPropagatingDispatcher$$anon$1.$anonfun$execute$1(ContextPropagatingDisptacher.scala:57)
    at akka.dispatch.TaskInvocation.run(AbstractDispatcher.scala:48)
    at akka.dispatch.ForkJoinExecutorConfigurator$AkkaForkJoinTask.exec(ForkJoinExecutorConfigurator.scala:48)
    at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
    at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
    at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
    at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:175)

Steps to Reproduce

  1. Applied the changes to /etc/thehive/application.conf as writen in https://docs.thehive-project.org/thehive/operations/update/

This is an extract of my config file:

db.janusgraph {
  storage {
    backend: cql
    hostname: ["127.0.0.1"]
    cql {
      cluster-name: thp
      keyspace: thehive
      read-consistency-level: ONE
      write-consistency-level: ONE
    }
  }

 index {
      search {
        backend: lucene
        directory: /opt/thp/thehive/index
      }
    }
  // storage.backend: berkeleyje
  // storage.directory: /path/to/berkeleydb
  // berkeleyje.freeDisk: 200 
}

storage {

  provider: localfs
  localfs.directory: /opt/thp_data/files/thehive
  // provider: hdfs
  // hdfs {
  //   root: "hdfs://localhost:10000" # namenode server hostname
  //   location: "/thehive"           # location inside HDFS
  //   username: thehive              # file owner
  // }
}
  1. Restarted The Hive, everything works properly
  2. Installed the stable verion following this guide https://docs.thehive-project.org/thehive/installation-and-configuration/installation/step-by-step-guide/ , I executed only the steps on the "The Hive" section, since everything else seems to be already in place: 
echo 'deb https://deb.thehive-project.org release main' | sudo tee -a /etc/apt/sources.list.d/thehive-project.list
sudo apt-get update
sudo apt-get install thehive4

4.-The hive does not come up again, everytime I restart the service (which appear as "running" as per "service thehive status") I get the same error stated above.

Possible Solutions

I did some reading, and it appears to be related to a variable that is set on the database, and can't be overwriten by the configuration file, since the warning stated the "harcoded" value of scalligraph can't be changed, and the error does refer scalligraph usage. I found a reference to change the GLOBAL_OFFLINE values, but I Tought it was better to ask. https://docs.janusgraph.org/configs/#global-configuration

Complementary information

application.log.txt install_application.log.txt

I added the logs from a service restart at application.log.txt, and full logs from the install at install_application.og.txt

marpoe commented 2 years ago

Unfortunately I faced exactely the same issue. Coming from "4.0.2-1".

nadouani commented 2 years ago

@huesitosloco can you please add this property to your application.conf file

db.janusgraph.forceDropAndRebuildIndex: true

Restart TheHive, it will rebuild the index and index the data. Once done, you can remove the db.janusgraph.forceDropAndRebuildIndex config or set it to false

mihai1978 commented 2 years ago

I have the same issue and the workarround with db.janusgraph.forceDropAndRebuildIndex is not working for me.

nadouani commented 2 years ago

Hello @huesitosloco @mihai1978 we've investigated the issue with @marpoe and there is in fact a bug that needs to be fixed. Adding this issue to 4.1.17

mihai1978 commented 2 years ago

I see, thank you for your answer.