search

DozerDB / dozerdb-core

DozerDB Plugin Core Project
GNU General Public License v3.0
27 stars 4 forks source link

APOC plugin causing error #22

Closed awhillas closed 3 months ago

awhillas commented 3 months ago

hi, So when I run the following docker-compose config

version: "3"
services:
  dozerdb:
    container_name: neo4j
    image: graphstack/dozerdb:5.22.0
    ports:
      - 7474:7474
      - 7687:7687
    environment:
      - NEO4J_AUTH=neo4j/password
      - NEO4J_apoc_export_file_enabled=true
      - NEO4J_apoc_import_file_enabled=true
      - NEO4J_apoc_import_file_use__neo4j__config=true
      - NEO4J_PLUGINS=["apoc"]
      - NEO4J_dbms_security_procedures_unrestricted='*'
    volumes:
      - ./data/neo4j/data:/data
      - ./data/neo4j/logs:/logs
      - ./data/neo4j/import:/var/lib/neo4j/import
      - ./data/neo4j/plugins:/plugins

I get the following error not when the - NEO4J_PLUGINS=["apoc"] line is removed. I guess because the latest version of that plugin is not compatible with DozerDB?

neo4j     | 2024-08-13 21:25:41.742+0000 INFO  GDS compatibility: for Neo4j 5.16 -- not available, for Neo4j 5.17 -- not available, for Neo4j 5.18 -- not available, for Neo4j 5.19 -- not available, for Neo4j 5.20 -- not available, for Neo4j 5.21 -- available, for Neo4j 5.22 -- not available, selected: Neo4j 5.21
neo4j     | 2024-08-13 21:25:41.763+0000 ERROR Failed to start Neo4j on 0.0.0.0:7474.
neo4j     | java.lang.RuntimeException: Error starting Neo4j database server at /data/databases
neo4j     |     at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:280) ~[neo4j-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.build(DatabaseManagementServiceFactory.java:215) ~[neo4j-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.server.DozerDbBootstrapper.createNeo(DozerDbBootstrapper.java:46) ~[dozerdb-core-5.21.2-1.0.0-alpha.1.jar:5.21.2-1.0.0-alpha.1]
neo4j     |     at org.neo4j.server.NeoBootstrapper.start(NeoBootstrapper.java:184) [neo4j-5.21.2.jar:5.21.2-1.0.0-alpha.1]
neo4j     |     at org.neo4j.server.NeoBootstrapper.start(NeoBootstrapper.java:99) [neo4j-5.21.2.jar:5.21.2-1.0.0-alpha.1]
neo4j     |     at org.neo4j.server.DozerDbEntryPoint.main(DozerDbEntryPoint.java:48) [dozerdb-core-5.21.2-1.0.0-alpha.1.jar:5.21.2-1.0.0-alpha.1]
neo4j     | Caused by: org.neo4j.kernel.lifecycle.LifecycleException: Component 'com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter@1d55dd21' failed to initialize. Please see the attached cause exception "com.neo4j.metrics.MetricsManager".
neo4j     |     at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:334) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.kernel.extension.AbstractExtensions.init(AbstractExtensions.java:72) ~[neo4j-kernel-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:317) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:85) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:271) ~[neo4j-5.21.2.jar:5.21.2]
neo4j     |     ... 5 more
neo4j     | Caused by: java.lang.NoClassDefFoundError: com/neo4j/metrics/MetricsManager
neo4j     |     at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.getMetricsRegister(GdsMetricsExtension.java:106) ~[graph-data-science.jar:?]
neo4j     |     at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.init(GdsMetricsExtension.java:79) ~[graph-data-science.jar:?]
neo4j     |     at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:317) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.kernel.extension.AbstractExtensions.init(AbstractExtensions.java:72) ~[neo4j-kernel-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:317) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:85) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:271) ~[neo4j-5.21.2.jar:5.21.2]
neo4j     |     ... 5 more
neo4j     |     Suppressed: org.neo4j.kernel.lifecycle.LifecycleException: Exception during graceful attempt to shutdown partially initialized component. Please use non suppressed exception to see original component failure.
neo4j     |             at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:324) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |             at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |             at org.neo4j.kernel.extension.AbstractExtensions.init(AbstractExtensions.java:72) ~[neo4j-kernel-5.21.2.jar:5.21.2]
neo4j     |             at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:317) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |             at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |             at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:85) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |             at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:271) ~[neo4j-5.21.2.jar:5.21.2]
neo4j     |             at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.build(DatabaseManagementServiceFactory.java:215) ~[neo4j-5.21.2.jar:5.21.2]
neo4j     |             at org.neo4j.server.DozerDbBootstrapper.createNeo(DozerDbBootstrapper.java:46) ~[dozerdb-core-5.21.2-1.0.0-alpha.1.jar:5.21.2-1.0.0-alpha.1]
neo4j     |             at org.neo4j.server.NeoBootstrapper.start(NeoBootstrapper.java:184) [neo4j-5.21.2.jar:5.21.2-1.0.0-alpha.1]
neo4j     |             at org.neo4j.server.NeoBootstrapper.start(NeoBootstrapper.java:99) [neo4j-5.21.2.jar:5.21.2-1.0.0-alpha.1]
neo4j     |             at org.neo4j.server.DozerDbEntryPoint.main(DozerDbEntryPoint.java:48) [dozerdb-core-5.21.2-1.0.0-alpha.1.jar:5.21.2-1.0.0-alpha.1]
neo4j     |     Caused by: java.lang.NoClassDefFoundError: com/neo4j/metrics/MetricsManager
neo4j     |             at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.getMetricsRegister(GdsMetricsExtension.java:106) ~[graph-data-science.jar:?]
neo4j     |             at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.shutdown(GdsMetricsExtension.java:98) ~[graph-data-science.jar:?]
neo4j     |             at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:322) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |             ... 11 more
neo4j     |     Caused by: java.lang.ClassNotFoundException: com.neo4j.metrics.MetricsManager
neo4j     |             at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.getMetricsRegister(GdsMetricsExtension.java:106) ~[graph-data-science.jar:?]
neo4j     |             at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.shutdown(GdsMetricsExtension.java:98) ~[graph-data-science.jar:?]
neo4j     |             at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:322) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |             ... 11 more
neo4j     | Caused by: java.lang.ClassNotFoundException: com.neo4j.metrics.MetricsManager
neo4j     |     at jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:641) ~[?:?]
neo4j     |     at jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188) ~[?:?]
neo4j     |     at java.lang.ClassLoader.loadClass(ClassLoader.java:525) ~[?:?]
neo4j     |     at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.getMetricsRegister(GdsMetricsExtension.java:106) ~[graph-data-science.jar:?]
neo4j     |     at com.neo4j.gds.metrics.GdsMetricsExtension$GdsMetricLifecycleAdapter.init(GdsMetricsExtension.java:79) ~[graph-data-science.jar:?]
neo4j     |     at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:317) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.kernel.extension.AbstractExtensions.init(AbstractExtensions.java:72) ~[neo4j-kernel-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.kernel.lifecycle.LifeSupport$LifecycleInstance.init(LifeSupport.java:317) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.kernel.lifecycle.LifeSupport.init(LifeSupport.java:56) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.kernel.lifecycle.LifeSupport.start(LifeSupport.java:85) ~[neo4j-common-5.21.2.jar:5.21.2]
neo4j     |     at org.neo4j.graphdb.facade.DatabaseManagementServiceFactory.startDatabaseServer(DatabaseManagementServiceFactory.java:271) ~[neo4j-5.21.2.jar:5.21.2]
neo4j     |     ... 5 more
neo4j     | 2024-08-13 21:25:41.777+0000 INFO  Neo4j Server shutdown initiated by request
neo4j     | 2024-08-13 21:25:41.779+0000 INFO  Stopped.
awhillas commented 3 months ago

Getting no joy with the OpenGDS 2.8.0-alpha01 you link to on your site.

org.neo4j.kernel.extension.FailedToBuildExtensionException: Failed to build kernel extension Extension:OpenGraphDataScienceExtension[gds.open] because it is compiled with a reference to a class, method, or field, that is not in the class path: 'GDS 2.8.0-alpha01 is not compatible with Neo4j version: 5.22.0'. The most common cause of this problem, is that Neo4j has been upgraded without also upgrading all installed extensions, such as APOC. Make sure that all of your extensions are build against your specific version of Neo4j.

Perhaps just publish a list of compatible plugins with the versions you have tested? Stability is more important than being the latest version of Neo4j. Especially if your just one guy and don't have CI/CD setup for your builds.

Also, is there some way that one can specify the version of the "apoc" plugin? (Can't find a concise explanation of what it does, functions lib?)

awhillas commented 3 months ago

OK, so I didn't notice you mention the GDS plugin you compiled to be compatible with DozerDB was for Neo4j Core 5.16, so I reverted back to graphstack/dozerdb:5.16 and d/l'ed the apoc plugin that was compatible with that version of Neo4j manually (I don't understand why the NEO4J_PLUGINS option isn't version sensitive or have an option to specify that) and now its working. Will need to figure out a way to automate all these manual steps in a Dockerfile, built on top of yours I guess. Hope this journey helps someone else with the same woes.

jmsuhy commented 3 months ago

If you have any ideas let me know - If we have too, we could re-implement the plugins functionality to only allow compatible plugins - but I would think that should be something Neo4j would want to do as it affects community edition.

I like the idea of being able to define a version of specific plugins. I am adding this as a topic to discuss for the roadmap.

awhillas commented 3 months ago

Yeah, might be a good idea. Seems like a bug if you specify NEO4J_PLUGINS and it downloads plugins which put it in a broken state.

awhillas commented 3 months ago

I guess something like NEO4J_PLUGINS=['apoc:5.16'] would work?

awhillas commented 3 months ago

Awesome work by the way. Game changer for me and my personal projects