search

torodb / stampede

The ToroDB solution to provide better analytics on top of MongoDB and make it easier to migrate from MongoDB to SQL
https://www.torodb.com/stampede/
GNU Affero General Public License v3.0
1.77k stars 120 forks source link

Using mysql backend incorrectly calling backend-common jar file #227

Closed spdawg11 closed 5 years ago

spdawg11 commented 5 years ago

Hi - testing out torodb stampede using latest version of tordb-stampede with a MySQL backend: 

OS> wget "https://www.torodb.com/download/torodb-stampede-latest.tar.bz2"
OS> torodb-stampede --version
0.50.3

And getting the following error when trying to start:

OS> ./torodb-stampede --log4j2-file /tmp/log4j2.xml
12:01:50.511 INFO  LIFECYCLE  'StampedeService-STARTING'     c.t.s.StampedeService          - Starting up ToroDB Stampede
12:01:50.723 INFO  BACKEND    'StampedeService-STARTING'     c.t.b.m.MySqlDbBackend         - Configured MySQL backend at localhost:3306
12:01:52.300 INFO  BACKEND    'MySqlDbBackend-STARTING'      c.t.b.AbstractDbBackendService - Created pool session with size 28 and level TRANSACTION_REPEATABLE_READ
12:01:52.327 INFO  BACKEND    'MySqlDbBackend-STARTING'      c.t.b.AbstractDbBackendService - Created pool system with size 1 and level TRANSACTION_REPEATABLE_READ
12:01:52.335 INFO  BACKEND    'MySqlDbBackend-STARTING'      c.t.b.AbstractDbBackendService - Created pool cursors with size 1 and level TRANSACTION_REPEATABLE_READ
12:01:52.496 INFO  BACKEND    'MySqlBackendBundle-STARTING'  c.t.b.m.AbstractSchemaUpdater  - Schema 'torodb' found. Checking it...
12:01:52.508 DEBUG LIFECYCLE  'main'                         c.t.s.Main                     - Fatal error on initialization
java.lang.IllegalStateException: Expected the service StampedeService [FAILED] to be TERMINATED, but the service has FAILED
        at com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:330) ~[guava-21.0.jar:?]
        at com.google.common.util.concurrent.AbstractService.awaitTerminated(AbstractService.java:293) ~[guava-21.0.jar:?]
        at com.google.common.util.concurrent.AbstractIdleService.awaitTerminated(AbstractIdleService.java:191) ~[guava-21.0.jar:?]
        at com.torodb.stampede.Main.main(Main.java:270) [torodb-stampede-1.0.0.jar:?]
Caused by: java.lang.IllegalStateException: Expected the service MySqlBackendBundle [FAILED] to be RUNNING, but the service has FAILED
        at com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:330) ~[guava-21.0.jar:?]
        at com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:266) ~[guava-21.0.jar:?]
        at com.google.common.util.concurrent.AbstractIdleService.awaitRunning(AbstractIdleService.java:175) ~[guava-21.0.jar:?]
        at com.torodb.stampede.StampedeService.startBundle(StampedeService.java:214) ~[stampede-service-1.0.0.jar:?]
        at com.torodb.stampede.StampedeService.startUp(StampedeService.java:100) ~[stampede-service-1.0.0.jar:?]
        at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[guava-21.0.jar:?]
        at com.google.common.util.concurrent.Callables$4.run(Callables.java:122) ~[guava-21.0.jar:?]
        at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_201]
Caused by: com.torodb.core.exceptions.InvalidDatabaseException: The schema 'torodb' does not contain the expected meta table 'kv'
        at com.torodb.backend.AbstractStructureInterface.checkMetaDataTables(AbstractStructureInterface.java:256) ~[backend-common-0.50.3.jar:?]
        at com.torodb.backend.meta.AbstractSchemaUpdater.checkSchema(AbstractSchemaUpdater.java:92) ~[backend-common-0.50.3.jar:?]
        at com.torodb.backend.meta.AbstractSchemaUpdater.checkOrCreate(AbstractSchemaUpdater.java:71) ~[backend-common-0.50.3.jar:?]
        at com.torodb.backend.meta.SchemaUpdater.checkOrCreate(SchemaUpdater.java:30) ~[backend-common-0.50.3.jar:?]
        at com.torodb.backend.ddl.DefaultStructureDdlOps.checkOrCreateMetaDataTables(DefaultStructureDdlOps.java:177) ~[backend-common-0.50.3.jar:?]
        at com.torodb.backend.ddl.DefaultDdlOperationExecutor.checkOrCreateMetaDataTables(DefaultDdlOperationExecutor.java:254) ~[backend-common-0.50.3.jar:?]
        at com.torodb.backend.AbstractBackendBundle.postDependenciesStartUp(AbstractBackendBundle.java:66) ~[backend-common-0.50.3.jar:?]
        at com.torodb.core.bundle.AbstractBundle.startUp(AbstractBundle.java:42) ~[core-0.50.3.jar:?]
        at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[guava-21.0.jar:?]
        at com.google.common.util.concurrent.Callables$4.run(Callables.java:122) ~[guava-21.0.jar:?]
        at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_201]
Fatal error while ToroDB was starting: The schema 'torodb' does not contain the expected meta table 'kv'
2019-03-12 12:01:52,536 Thread-1 WARN Unable to register Log4j shutdown hook because JVM is shutting down. Using SimpleLogger

Configuration file in use:

---
logging:
  level: "INFO"
  file: "/var/log/toro-stampede.log"
metricsEnabled: false
offHeapBuffer:
  enabled: false
  path: "/tmp/torodb-stampede1645648364423678119"
  rollCycle: "DAILY"
  maxFiles: 5
replication:
  replSetName: "toroDB"
  syncSource: "10.x.x.x:27017"
  ssl:
    enabled: false
    allowInvalidHostnames: false
    fipsMode: false
  auth:
    mode: "negotiate"
    user: torouser
    source: admin
  mongopassFile: "/root/.mongopass"
backend:
#  pool:
#    connectionPoolTimeout: 10000
#    connectionPoolSize: 30
  mysql:
    host: "localhost"
    port: 3306
    database: "torodb"
    user: "torouser"
    toropassFile: "/root/.toropass"
    applicationName: "toro"
    ssl: false

Do I have anything obviously wrong in my configuration so that backend-common is being called versus backend-mysql?

OS> ls -l /root/torodb-stampede-1.0.0/lib/com/torodb/engine/backend/backend*/0.50.3/
/root/torodb-stampede-1.0.0/lib/com/torodb/engine/backend/backend-common/0.50.3/:
total 316
-rw-r--r--. 1 torodb torodb 319888 Oct 24 14:20 backend-common-0.50.3.jar

/root/torodb-stampede-1.0.0/lib/com/torodb/engine/backend/backend-mysql/0.50.3/:
total 112
-rw-r--r--. 1 torodb torodb 111161 Oct 24 14:40 backend-mysql-0.50.3.jar

/root/torodb-stampede-1.0.0/lib/com/torodb/engine/backend/backend-postgresql/0.50.3/:
total 124
-rw-r--r--. 1 torodb torodb 126157 Oct 24 14:34 backend-postgresql-0.50.3.jar

Regards spdawg11

teoincontatto commented 5 years ago

Seems that your MySQL has a dirty torodb meta databases that ToroDB Stampede uses to store his tables. Dropping it should solve the issue.

spdawg11 commented 5 years ago

Thank you.

Have dropped the database but now toro complains it can't find the database.

./torodb-stampede --log4j2-file /tmp/log4j2.xml
03:52:26.635 INFO  LIFECYCLE  'StampedeService-STARTING'     c.t.s.StampedeService          - Starting up ToroDB Stampede
03:52:26.870 INFO  BACKEND    'StampedeService-STARTING'     c.t.b.m.MySqlDbBackend         - Configured MySQL backend at localhost:3306
03:52:28.435 DEBUG LIFECYCLE  'main'                         c.t.s.Main                     - Fatal error on initialization
java.lang.IllegalStateException: Expected the service StampedeService [FAILED] to be TERMINATED, but the service has FAILED
        at com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:330) ~[guava-21.0.jar:?]
        at com.google.common.util.concurrent.AbstractService.awaitTerminated(AbstractService.java:293) ~[guava-21.0.jar:?]
        at com.google.common.util.concurrent.AbstractIdleService.awaitTerminated(AbstractIdleService.java:191) ~[guava-21.0.jar:?]
        at com.torodb.stampede.Main.main(Main.java:270) [torodb-stampede-1.0.0.jar:?]
Caused by: java.lang.IllegalStateException: Expected the service MySqlBackendBundle [FAILED] to be RUNNING, but the service has FAILED
        at com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:330) ~[guava-21.0.jar:?]
        at com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:266) ~[guava-21.0.jar:?]
        at com.google.common.util.concurrent.AbstractIdleService.awaitRunning(AbstractIdleService.java:175) ~[guava-21.0.jar:?]
        at com.torodb.stampede.StampedeService.startBundle(StampedeService.java:214) ~[stampede-service-1.0.0.jar:?]
        at com.torodb.stampede.StampedeService.startUp(StampedeService.java:100) ~[stampede-service-1.0.0.jar:?]
        at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[guava-21.0.jar:?]
        at com.google.common.util.concurrent.Callables$4.run(Callables.java:122) ~[guava-21.0.jar:?]
        at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_201]
Caused by: java.lang.IllegalStateException: Expected the service MySqlDbBackend [FAILED] to be RUNNING, but the service has FAILED
        at com.google.common.util.concurrent.AbstractService.checkCurrentState(AbstractService.java:330) ~[guava-21.0.jar:?]
        at com.google.common.util.concurrent.AbstractService.awaitRunning(AbstractService.java:266) ~[guava-21.0.jar:?]
        at com.google.common.util.concurrent.AbstractIdleService.awaitRunning(AbstractIdleService.java:175) ~[guava-21.0.jar:?]
        at com.torodb.backend.AbstractBackendBundle.postDependenciesStartUp(AbstractBackendBundle.java:55) ~[backend-common-0.50.3.jar:?]
        at com.torodb.core.bundle.AbstractBundle.startUp(AbstractBundle.java:42) ~[core-0.50.3.jar:?]
        at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[guava-21.0.jar:?]
        at com.google.common.util.concurrent.Callables$4.run(Callables.java:122) ~[guava-21.0.jar:?]
        at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_201]
Caused by: com.torodb.core.exceptions.SystemException: Unknown database 'torodb'
        at com.torodb.backend.mysql.driver.OfficialMySqlDriver.getConfiguredDataSource(OfficialMySqlDriver.java:83) ~[backend-mysql-0.50.3.jar:?]
        at com.torodb.backend.mysql.MySqlDbBackend.getConfiguredDataSource(MySqlDbBackend.java:60) ~[backend-mysql-0.50.3.jar:?]
        at com.torodb.backend.AbstractDbBackendService.createPooledDataSource(AbstractDbBackendService.java:168) ~[backend-common-0.50.3.jar:?]
        at com.torodb.backend.AbstractDbBackendService.startUp(AbstractDbBackendService.java:111) ~[backend-common-0.50.3.jar:?]
        at com.google.common.util.concurrent.AbstractIdleService$DelegateService$1.run(AbstractIdleService.java:62) ~[guava-21.0.jar:?]
        at com.google.common.util.concurrent.Callables$4.run(Callables.java:122) ~[guava-21.0.jar:?]
        at java.lang.Thread.run(Thread.java:748) ~[?:1.8.0_201]
Fatal error while ToroDB was starting: Unknown database 'torodb'
2019-03-13 15:52:28,464 Thread-1 WARN Unable to register Log4j shutdown hook because JVM is shutting down. Using SimpleLogger

Does toro create the mysql database?

If not, and I create it myself - how does the expected schema get created (ie the 'kv' table contained in my error message)?

Regards spdawg11

teoincontatto commented 5 years ago

ToroDB Stampede does require the torodb database to be created. It should then create the meta tables itself.