Sleepycat dependecy issue

[DimitriosMaimaris]

Berkley is containing as a dependency sleepycat with version 7.5.11. By building the project on Github it leads to build failure since it cannot fetch that artifact. If the sleepycat version ( version 18) from maven central is being used in the parent pom the project is building with no issue but then it is not possible to open the JanusGraph from Gremlin console.

If you try to open the graph it has this error:

Caused by: com.sleepycat.je.VersionMismatchException: (JE 7.5.11) /Workspace/egeria/egeria-lineage-repositories/bufferGraph/berkeley Expected log version 15 or earlier but found 17 VERSION_MISMATCH: The existing log was written with a version of JE that is later than the running version of JE, the log cannot be read. Environment is invalid and must be closed. Couldn't open file /Users/Workspace/egeria/egeria-lineage-repositories/bufferGraph/berkeley/00000000.jdb

One solution is to load the artifact of sleepycat to the artifactory that is going to build the project or to find a way without loading the artifact to build the project and also be able to open the Graph without an error.

[planetf1]

The current state is that we use dependencyManagement to enforce a sleepcat dependency on a version available in maven central. This means anyone can take and build egeria without having to hunt down the cause of a build problem. This was addressed in June 2019 under https://github.com/odpi/egeria/issues/1116 and egeria builds cleanly.

We can't realistically revert to this version since it is not available to others.

From the error, the key to me is it says 'The existing log'. I would expect Janus to create the berkeleydb, and hence it's log file, on first. start - and that would always be with 7.5.11 (v15)

Can you confirm you are starting 'clean' - ie with no existing database in-place? Do you have a simple test case which demonstrates this issue? I'd like to try and reproduce it and consider trying other versions of sleepycat Can you check that 'mvn dependency:tree' is only listing one version of sleepcat? Can you confirm the environment where are running - are any other jars/modules added to the classpath dynamically - ie using LOADER_PATH, or other mechanisms, that could lead to having multiple versions of berkely present at runtime?

[planetf1]

Note that we use berkeley with janusgraph for the graph based repo - which is not showing this issue, so need to understand what is different.

Though loading to artifactory works for us - and even the official build, it would not for others - some developers may not have their own, so we'd have to tell them where to get it from - and the original repo has license challenges. Further since berkeley is the sensible default for our graph repo, I'd be loathe to require an extra step to make use of a local repository (which is already working).

It does seem the most likely cause is referring to a db created by an earlier version

[DimitriosMaimaris]

I did not say the application has issue. To debug the queries by running them to gremlin is the issue

Can you confirm you are starting 'clean' - ie with no existing database in-place?

No existing database with new

Do you have a simple test case which demonstrates this issue? I'd like to try and reproduce it and consider trying other versions of sleepycat

1. Build project as is from master.
   2. Delete the graph you have for local repo
   3. Instantiate egeria (only local repo if you want does not matter)
   4. If you started it and the folder of the graph is created stop egeria.
   5. Download janusgraph 0.4.0
   6. Under conf folder of janusgraph create a file egeria.properties Put inside by replacing the path to where you have omas-graph-repository storage.backend=berkeleyje index.search.backend=lucene index.search.directory=/Users///Projects/master/egeria/omas-graph-repository/searchindex storage.directory=/Users///Projects/master/egeria/omas-graph-repository/berkeley
   7. Go to bin of janusgraph folder(Egeria should be stopped) and run gremlin.sh
   8. After gremlin starts run graph =JanusGraphFactory.open("conf/egeria.properties")
   9. You will see the error

Can you check that 'mvn dependency:tree' is only listing one version of sleepcat?

dependency tree only shows version 18.*

Can you confirm the environment where are running - are any other jars/modules added to the classpath dynamically - ie using LOADER_PATH, or other mechanisms, that could lead to having multiple versions of berkely present at runtime?

Yes there is a jar open-lineage-janus-connector added as an external library.

[planetf1]

Thanks for the info. Very helpful. Will give it a go. I suspect the issue is in the gremlin.sh execution. WIll take a look & see if I can come up with any ideas

[DimitriosMaimaris]

Cool thanks

[planetf1]

I do see that error here - but we are going outside what we build with egeria here -- so by running a completely distinct janus environment you are indeed picking up a different version of berkeley.

Is the scenario you describe purely for debugging?

Another - perhaps better option is to generally run a gremlin server, then the client is only communicating via a network api and we don't get into issues of library version differences.

You would get the same issue if in future janus updated some dependencies ?

As a workaround - which could be suitable for troubleshooting/developer use you could copy the library over ie:

cp ~/.m2/repository/com/sleepycat/je/18.3.12/je-18.3.12.jar ./lib/je-18.3.12.jar

I was then able to access the db

➜  janusgraph-0.4.1-hadoop2 bin/gremlin.sh
WARNING: An illegal reflective access operation has occurred
WARNING: Illegal reflective access by org.codehaus.groovy.vmplugin.v7.Java7$1 (file:/Users/jonesn/IdeaProjects/janus/janusgraph-0.4.1-hadoop2/lib/groovy-2.5.7-indy.jar) to constructor java.lang.invoke.MethodHandles$Lookup(java.lang.Class,int)
WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.vmplugin.v7.Java7$1
WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations
WARNING: All illegal access operations will be denied in a future release

         \,,,/
         (o o)
-----oOOo-(3)-oOOo-----
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/Users/jonesn/IdeaProjects/janus/janusgraph-0.4.1-hadoop2/lib/slf4j-log4j12-1.7.12.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/Users/jonesn/IdeaProjects/janus/janusgraph-0.4.1-hadoop2/lib/logback-classic-1.1.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
plugin activated: tinkerpop.server
plugin activated: tinkerpop.tinkergraph
14:54:37 WARN  org.apache.hadoop.util.NativeCodeLoader  - Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
plugin activated: tinkerpop.hadoop
plugin activated: tinkerpop.spark
plugin activated: tinkerpop.utilities
plugin activated: janusgraph.imports
gremlin> graph =JanusGraphFactory.open("conf/egeria.properties")
==>standardjanusgraph[berkeleyje:/Users/jonesn/src/egeria/cocoMDS2-graph-repository/berkeley]
gremlin>

So in summary we could consider a) Updating any docs on debugging to copy library directly b) consider focussing more on a seperate gremlin server - easier to debug

I would suggest no change is needed to the dependency?

Comments @grahamwallis ?

[planetf1]

@DimitriosMaimaris Does this address your issue? Are those updates appropriate for your developer guide docs? Any further investigation needed? If not I'll assign to you later to write up the new instructions (I've not seen the original, or I'd be happy to test and update)

[DimitriosMaimaris]

Lets go that way yes I agree.