search

JanusGraph / janusgraph

JanusGraph: an open-source, distributed graph database
https://janusgraph.org
Other
5.27k stars 1.16k forks source link

Could not instantiate implementation: org.janusgraph.diskstorage.solr.SolrIndex #2889

Closed buck-ross closed 2 years ago

buck-ross commented 2 years ago

Description

When running JanuceGraph using Apache Cassandra & Apache Solr, JanusGraph spits out the following error:

java.lang.RuntimeException: GraphFactory could not instantiate this Graph implementation [class org.janusgraph.core.JanusGraphFactory]      at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:84)      at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:72)      at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:106)      at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.addGraph(DefaultGraphManager.java:63)      at java.util.LinkedHashMap.forEach(LinkedHashMap.java:684)      at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.(DefaultGraphManager.java:58)      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)      at java.lang.reflect.Constructor.newInstance(Constructor.java:423)      at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.(ServerGremlinExecutor.java:84)      at org.apache.tinkerpop.gremlin.server.GremlinServer.(GremlinServer.java:124)      at org.apache.tinkerpop.gremlin.server.GremlinServer.(GremlinServer.java:87)      at org.janusgraph.graphdb.server.JanusGraphServer.start(JanusGraphServer.java:85)      at org.janusgraph.graphdb.server.JanusGraphServer.main(JanusGraphServer.java:53) Caused by: java.lang.reflect.InvocationTargetException      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)      at java.lang.reflect.Method.invoke(Method.java:498)      at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:80)      ... 14 more Caused by: java.lang.IllegalArgumentException: Could not instantiate implementation: org.janusgraph.diskstorage.solr.SolrIndex      at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:79)      at org.janusgraph.diskstorage.Backend.getImplementationClass(Backend.java:525)      at org.janusgraph.diskstorage.Backend.getIndexes(Backend.java:509)      at org.janusgraph.diskstorage.Backend.(Backend.java:239)      at org.janusgraph.graphdb.configuration.GraphDatabaseConfiguration.getBackend(GraphDatabaseConfiguration.java:1355)      at org.janusgraph.graphdb.database.StandardJanusGraph.(StandardJanusGraph.java:184)      at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:176)      at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:147)      at org.janusgraph.core.JanusGraphFactory.open(JanusGraphFactory.java:127)      ... 19 more Caused by: java.lang.reflect.InvocationTargetException      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)      at java.lang.reflect.Constructor.newInstance(Constructor.java:423)      at org.janusgraph.util.system.ConfigurationUtil.instantiate(ConfigurationUtil.java:73)      ... 27 more Caused by: java.lang.NoSuchMethodError: org.noggit.ObjectBuilder.getValStrict()Ljava/lang/Object;      at org.apache.solr.common.util.Utils.fromJSON(Utils.java:319)      at org.apache.solr.common.util.Utils.fromJSON(Utils.java:305)      at org.apache.solr.common.cloud.ClusterState.load(ClusterState.java:245)      at org.apache.solr.common.cloud.ZkStateReader.refreshLegacyClusterState(ZkStateReader.java:637)      at org.apache.solr.common.cloud.ZkStateReader.createClusterStateWatchersAndUpdate(ZkStateReader.java:511)      at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.getZkStateReader(ZkClientClusterStateProvider.java:177)      at org.apache.solr.client.solrj.impl.ZkClientClusterStateProvider.connect(ZkClientClusterStateProvider.java:161)      at org.apache.solr.client.solrj.impl.BaseCloudSolrClient.connect(BaseCloudSolrClient.java:349)      at org.janusgraph.diskstorage.solr.SolrIndex.(SolrIndex.java:296)      ... 32 more

The stack trace doesn't seem to yield too much useful information (at least from my perspective), but the line java.lang.IllegalArgumentException: Could not instantiate implementation: org.janusgraph.diskstorage.solr.SolrIndex seems like a strong indicator that this id due to a failed connection to the Apache SolrCloud instance.

Reproducing with Containers

Below is a script to reproduce the above error (I'm using podman's "pod" feature to make all of the containers share a common networking namespace. With docker, this can be accomplished with --link):

# Create fresh directories for the volume mounts:
mkdir -p ./volumes/{solr,cassandra,zk}
podman unshare chown -Rv 8983:8983 ./volumes/solr

# Provision the pod container:
podman pod create --name test -p 2181:2181 -p 8182:8182 -p 8983:8983 -p 9042:9042

# Deploy Zookeeper to support SolrCloud:
podman run -d --pod test --name zk -v ./volumes/zk:/data:Z zookeeper:latest

# Deploy SolrCloud:
podman run -d --pod test --name solr -e 'ZK_HOST=127.0.0.1:2181' -v ./volumes/solr:/var/solr:Z solr:8

# Deploy Cassandra:
podman run -d --pod test --name cassandra -v ./volumes/cassandra:/var/lib/cassandra:Z cassandra:3

# Deploy the JanusGraph instance:
podman run -d --pod test --name janusgraph \
    -e 'janusgraph.index.search.backend=solr' \
    -e 'janusgraph.index.search.solr.mode=cloud' \
    -e 'janusgraph.index.search.solr.zookeeper-url=localhost:2181' \
    -e 'janusgraph.storage.backend=cql' \
    -e 'janusgraph.cache.db-cache=true' \
    -e 'janusgraph.storage.hostname=127.0.0.1' \
    janusgraph/janusgraph:latest

NOTE: You will need to wait for Cassandra & Solr to become operational before deploying JanusGraph, otherwise it will fail to start.

Once the JanusGraph instance has finished starting up, checking it's logs with podman logs janusgraph will yield the above error message, along with a following message about being unable to create GremlinScriptEngine for gremlin-groovy, which I believe is directly caused by the failure to connect to Solr.

You can then cleanup all services / data from this reproduction with:

podman pod stop test
podman pod rm test
rm -rv ./volumes

Thank you for your time!

buck-ross commented 2 years ago

I have just confirmed that neither of the errors I mentioned occurs when the environment variable janusgraph.index.search.backend is set to lucene, which seems to confirm that this is an issue with connecting to SolrCloud.

mad commented 2 years ago

Fast fix just remove this jar $JANUSGRAPH/lib/noggit-0.6.jar

buck-ross commented 2 years ago

@mad Thanks for the quick response! That seems to have worked, and JanusGraph now outputs that it is able to connect to Zookeeper without any issue, but I'm a little concerned that I have no idea what I just did to my deployment. If it's not too much trouble, would you mind explaining to me what noggit-0.8.jar was doing, and why removing it has fixed this error? I just want to make sure this doesn't cause any other problems later on down the road.

vtslab commented 2 years ago

https://github.com/JanusGraph/janusgraph/search?q=noggit As you see, noggit is supposed to be excluded from janusgraph-solr, but it comes in again through janus-graph-driver. So, there should be no impact as long as you do not use the client and server from the same lib/ folder.

buck-ross commented 2 years ago

Thanks so much to both of you for the help in figuring this issue out! Since there doesn't seem to be much immediate drive to fix this in the official docker image, I've gone ahead and created my own repository at buckross/janusgraph-sin-noggit, which excludes the noggit library.

I'm going to go ahead and close this issue for now. If anyone out there feels strongly that this needs to be further addressed, just @ me, and I'll re-open.

/close