search

DozerDB / dozerdb-core

DozerDB Plugin Core Project
GNU General Public License v3.0
27 stars 4 forks source link

Docker Container exits with failure when importing GDS plugin. #17

Closed tomksck closed 4 months ago

tomksck commented 4 months ago

When using the following docker-compose.yml and Dockerfile the Dozerdb Container won't start. When replacing the plugin with apoc or any other plugin, the container starts up as expected.

docker-compose.yml:

services:
  neo4j:
    build: .
    ports:
      - "7474:7474"
      - "7687:7687"
    volumes:
      - $HOME/neo4j/data:/data
      - $HOME/neo4j/logs:/logs
      - $HOME/neo4j/import:/var/lib/neo4j/import
      - $HOME/neo4j/plugins:/plugins
    environment:
      - NEO4J_AUTH=neo4j/password
      - NEO4J_PLUGINS=["graph-data-science"]
      - NEO4J_apoc_export_file_enabled=true
      - NEO4J_apoc_import_file_enabled=true
      - NEO4J_dbms_security_procedures_unrestricted=*

Dockerfile:

FROM graphstack/dozerdb:5.20.0.0-alpha.1
EXPOSE 7474 7687
CMD ["neo4j"]

And I receive the following Docker output:

2024-07-08 13:22:06 neo4j-1  | Fetching versions.json for Plugin 'graph-data-science' from https://graphdatascience.ninja/versions.json
2024-07-08 13:22:06 neo4j-1  | Installing Plugin 'graph-data-science' from https://graphdatascience.ninja/neo4j-graph-data-science-2.7.0.jar to /plugins/graph-data-science.jar 
2024-07-08 13:22:08 neo4j-1  | Applying default values for plugin graph-data-science to neo4j.conf
2024-07-08 13:22:09 neo4j-1  | SLF4J(W): Class path contains multiple SLF4J providers.
2024-07-08 13:22:09 neo4j-1  | SLF4J(W): Found provider [org.slf4j.jul.JULServiceProvider@6c372fe6]
2024-07-08 13:22:09 neo4j-1  | SLF4J(W): Found provider [org.neo4j.server.logging.slf4j.SLF4JLogBridge@58594a11]
2024-07-08 13:22:09 neo4j-1  | SLF4J(W): See https://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2024-07-08 13:22:09 neo4j-1  | SLF4J(I): Actual provider is of type [org.slf4j.jul.JULServiceProvider@6c372fe6]
2024-07-08 13:22:09 neo4j-1  | Changed password for user 'neo4j'. IMPORTANT: this change will only take effect if performed before the database is started for the first time.
2024-07-08 13:22:11 neo4j-1  | 2024-07-08 11:22:11.273+0000 INFO  Logging config in use: File '/var/lib/neo4j/conf/user-logs.xml'
2024-07-08 13:22:11 neo4j-1  | 2024-07-08 11:22:11.282+0000 INFO  Starting...
2024-07-08 13:22:11 neo4j-1  | 2024-07-08 11:22:11.282+0000 INFO  
2024-07-08 13:22:11 neo4j-1  | 
2024-07-08 13:22:11 neo4j-1  | *****************************************************************************
2024-07-08 13:22:11 neo4j-1  |  *********************** Enhanced By DozerDB Plugin ***********************
2024-07-08 13:22:11 neo4j-1  | *****************************************************************************
2024-07-08 13:22:11 neo4j-1  | 
2024-07-08 13:22:11 neo4j-1  | 
2024-07-08 13:22:11 neo4j-1  | 2024-07-08 11:22:11.799+0000 INFO  This instance is ServerId{c490b4e6} (c490b4e6-f556-4679-b5bd-6b77518b2ebb)
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.308+0000 INFO  ======== Neo4j 5.20.0 ========
2024-07-08 13:22:12 neo4j-1  | SLF4J(W): Class path contains multiple SLF4J providers.
2024-07-08 13:22:12 neo4j-1  | SLF4J(W): Found provider [org.slf4j.jul.JULServiceProvider@7b14c61]
2024-07-08 13:22:12 neo4j-1  | SLF4J(W): Found provider [org.neo4j.server.logging.slf4j.SLF4JLogBridge@8fcc534]
2024-07-08 13:22:12 neo4j-1  | SLF4J(W): See https://www.slf4j.org/codes.html#multiple_bindings for an explanation.
2024-07-08 13:22:12 neo4j-1  | SLF4J(I): Actual provider is of type [org.slf4j.jul.JULServiceProvider@7b14c61]
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.370+0000 INFO  GDS license file: null
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.371+0000 INFO  GDS license state: unlicensed
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.376+0000 INFO  Progress tracking: enabled
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.377+0000 INFO  Memory usage guard: minimum estimate
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.380+0000 INFO  Building Graph Data Science extension...
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.383+0000 INFO  Register Graph Data Science...
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.384+0000 INFO  Graph Data Science registered.
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.385+0000 INFO  Register Metrics Facade...
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.385+0000 INFO  Metrics Facade registered.
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.386+0000 INFO  Register Model Catalog...
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.386+0000 INFO  Model Catalog registered.
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.387+0000 INFO  Register Task Registry Factory...
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.387+0000 INFO  Task Registry Factory registered.
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.388+0000 INFO  Register Task Store...
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.388+0000 INFO  Task Store registered.
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.388+0000 INFO  Register User Log Registry Factory...
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.389+0000 INFO  User Log Registry Factory registered.
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.389+0000 INFO  Graph Data Science extension built.
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.390+0000 INFO  Building Graph Data Science Extras extension...
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.390+0000 INFO  Register Graph Data Science Extras...
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.391+0000 INFO  Graph Data Science Extras registered.
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.391+0000 INFO  Graph Data Science Extras extension built.
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.565+0000 INFO  GDS compatibility: for Neo4j 5.15 -- not available, for Neo4j 5.16 -- not available, for Neo4j 5.17 -- not available, for Neo4j 5.18 -- not available, for Neo4j 5.19 -- not available, for Neo4j 5.20 -- available, for Neo4j 5.21 -- not available, selected: Neo4j 5.20
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.581+0000 ERROR Failed to start Neo4j on 0.0.0.0:7474.
2024-07-08 13:22:12 neo4j-1  | java.lang.RuntimeException: Error starting Neo4j database server at /data/databases
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:278) ~[neo4j-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.build(DatabaseManagementServiceFactory.java:213) ~[neo4j-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.server.DozerDbBootstrapper.createNeo(DozerDbBootstrapper.java:46) ~[dozerdb-core-5.20.0-1.0.0-alpha.1.jar:5.20.0-1.0.0-alpha.1]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.server.NeoBootstrapper.start(NeoBootstrapper.java:184) [neo4j-5.20.0.jar:5.20.0-1.0.0-alpha.1]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.server.NeoBootstrapper.start(NeoBootstrapper.java:99) [neo4j-5.20.0.jar:5.20.0-1.0.0-alpha.1]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.server.DozerDbEntryPoint.main(DozerDbEntryPoint.java:48) [dozerdb-core-5.20.0-1.0.0-alpha.1.jar:5.20.0-1.0.0-alpha.1]
2024-07-08 13:22:12 neo4j-1  | Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter@22b82ddf' failed to initialize. Please see the attached cause exception "com.neo4j.metrics.MetricsManager".
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:334) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.extension.AbstractExtensions.init(AbstractExtensions.java:72) ~[neo4j-kernel-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:317) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:85) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:269) ~[neo4j-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  ... 5 more
2024-07-08 13:22:12 neo4j-1  | Caused by: java.lang.NoClassDefFoundError: com/neo4j/metrics/MetricsManager
2024-07-08 13:22:12 neo4j-1  |  at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.getMetricsRegister(GdsMetricsExtension.java:106) ~[graph-data-science.jar:?]
2024-07-08 13:22:12 neo4j-1  |  at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.init(GdsMetricsExtension.java:79) ~[graph-data-science.jar:?]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:317) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.extension.AbstractExtensions.init(AbstractExtensions.java:72) ~[neo4j-kernel-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:317) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:85) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:269) ~[neo4j-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  ... 5 more
2024-07-08 13:22:12 neo4j-1  |  Suppressed: org.neo4j.kernel.lifecycle.LifecycleException: Exception during graceful attempt to shutdown partially initialized component. Please use non suppressed exception to see original component failure.
2024-07-08 13:22:12 neo4j-1  |          at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:324) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |          at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |          at org.neo4j.kernel.extension.AbstractExtensions.init(AbstractExtensions.java:72) ~[neo4j-kernel-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |          at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:317) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |          at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |          at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:85) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |          at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:269) ~[neo4j-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |          at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.build(DatabaseManagementServiceFactory.java:213) ~[neo4j-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |          at org.neo4j.server.DozerDbBootstrapper.createNeo(DozerDbBootstrapper.java:46) ~[dozerdb-core-5.20.0-1.0.0-alpha.1.jar:5.20.0-1.0.0-alpha.1]
2024-07-08 13:22:12 neo4j-1  |          at org.neo4j.server.NeoBootstrapper.start(NeoBootstrapper.java:184) [neo4j-5.20.0.jar:5.20.0-1.0.0-alpha.1]
2024-07-08 13:22:12 neo4j-1  |          at org.neo4j.server.NeoBootstrapper.start(NeoBootstrapper.java:99) [neo4j-5.20.0.jar:5.20.0-1.0.0-alpha.1]
2024-07-08 13:22:12 neo4j-1  |          at org.neo4j.server.DozerDbEntryPoint.main(DozerDbEntryPoint.java:48) [dozerdb-core-5.20.0-1.0.0-alpha.1.jar:5.20.0-1.0.0-alpha.1]
2024-07-08 13:22:12 neo4j-1  |  Caused by: java.lang.NoClassDefFoundError: com/neo4j/metrics/MetricsManager
2024-07-08 13:22:12 neo4j-1  |          at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.getMetricsRegister(GdsMetricsExtension.java:106) ~[graph-data-science.jar:?]
2024-07-08 13:22:12 neo4j-1  |          at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.shutdown(GdsMetricsExtension.java:98) ~[graph-data-science.jar:?]
2024-07-08 13:22:12 neo4j-1  |          at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:322) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |          ... 11 more
2024-07-08 13:22:12 neo4j-1  |  Caused by: java.lang.ClassNotFoundException: com.neo4j.metrics.MetricsManager
2024-07-08 13:22:12 neo4j-1  |          at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.getMetricsRegister(GdsMetricsExtension.java:106) ~[graph-data-science.jar:?]
2024-07-08 13:22:12 neo4j-1  |          at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.shutdown(GdsMetricsExtension.java:98) ~[graph-data-science.jar:?]
2024-07-08 13:22:12 neo4j-1  |          at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:322) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |          ... 11 more
2024-07-08 13:22:12 neo4j-1  | Caused by: java.lang.ClassNotFoundException: com.neo4j.metrics.MetricsManager
2024-07-08 13:22:12 neo4j-1  |  at jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) ~[?:?]
2024-07-08 13:22:12 neo4j-1  |  at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) ~[?:?]
2024-07-08 13:22:12 neo4j-1  |  at java.lang.ClassLoader.loadClass(ClassLoader.java:525) ~[?:?]
2024-07-08 13:22:12 neo4j-1  |  at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.getMetricsRegister(GdsMetricsExtension.java:106) ~[graph-data-science.jar:?]
2024-07-08 13:22:12 neo4j-1  |  at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.init(GdsMetricsExtension.java:79) ~[graph-data-science.jar:?]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:317) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.extension.AbstractExtensions.init(AbstractExtensions.java:72) ~[neo4j-kernel-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:317) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:85) ~[neo4j-common-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:269) ~[neo4j-5.20.0.jar:5.20.0]
2024-07-08 13:22:12 neo4j-1  |  ... 5 more
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.584+0000 INFO  Neo4j Server shutdown initiated by request
2024-07-08 13:22:12 neo4j-1  | 2024-07-08 11:22:12.584+0000 INFO  Stopped.
jmsuhy commented 4 months ago

We are looking into this and will update the ticket once we identify the issue. If we need anything else, I’ll let you know.

jmsuhy commented 4 months ago

It appears that Neo4j has made the version of the GDS plugin require Neo4j Enterprise now.
This error should also happen with Neo4j Community Edition standalone without DozerDB.

NEO4J_PLUGINS=["graph-data-science"]
Notice: Caused by: java.lang.ClassNotFoundException: com.neo4j.metrics.MetricsManager

We have a team meeting tomorrow, and will discuss how we want to tackle this.

jmsuhy commented 4 months ago

FYI - https://github.com/neo4j/graph-data-science https://github.com/neo4j/graph-data-science/commit/02c77c85055a38a54146957621d721f64e497a95

So it seems that 5.21 has compatibility with openGDS.

jmsuhy commented 4 months ago

Another good reference - it seems Neo4j does not want to make it easy to use openGDS. Someone has created a build script - worst case, we can make it part of our build process as well.

https://github.com/neo4j/graph-data-science/issues/275

jmsuhy commented 4 months ago

@tomksck - It looks like OpenGDS works fine - you would have to manually add it to your $HOME/neo4j/plugins folder. We are setting up a build for the openGDS plugin until Neo4j gets the official releases out.

You can also grab a build from https://github.com/JohT/open-graph-data-science-packaging if you want to test yourself.

Remember to remove the enterprise graph-data-science plugin from your $HOME/neo4j/plugins first.

I'm going to close this ticket as its for trying to get the enterprise graph data science plugin working.

Let me know if everything works for you.

jmsuhy commented 4 months ago

FYI - if you want to test on the new DozerDB version for Neo4j Core 5.21.2 - you can get the newest OpenGDS (Open Graph Data Science) jar we packaged and put it in your $HOME/neo4j/plugins folder.

https://dist.dozerdb.org/plugins/open-gds/open-gds-2.8.0-alpha01.jar