search

ArcadeData / arcadedb

ArcadeDB Multi-Model Database, one DBMS that supports SQL, Cypher, Gremlin, HTTP/JSON, MongoDB and Redis. ArcadeDB is a conceptual fork of OrientDB, the first Multi-Model DBMS. ArcadeDB supports Vector Embeddings.
https://arcadedb.com
Apache License 2.0
494 stars 60 forks source link

Gremlin Server Plugin Persistent Volume Claim #1065

Closed humutkazan closed 1 year ago

humutkazan commented 1 year ago

ArcadeDB Version: <23.3.1>

JDK Version: <OpenJDK 64-Bit Server VM Temurin-11.0.18+10>

Expected behavior

Persist gremlin server output

Actual behavior

Can't get it to persist

Hi

I am trying to deploy arcadedb with gremlin server plugin. I used statefulset.yaml from this repo with a little bit of change. Since gremlin.arcadedb.directory is set to databases/graph I want to persist that directory but haven't been able to achieve it yet. apart from mnt/data0 I also added home/arcadedb/databases as mountPath but it throws this error

` █████╗ ██████╗ ██████╗ █████╗ ██████╗ ███████╗██████╗ ██████╗ ██╔══██╗██╔══██╗██╔════╝██╔══██╗██╔══██╗██╔════╝██╔══██╗██╔══██╗ ███████║██████╔╝██║ ███████║██║ ██║█████╗ ██║ ██║██████╔╝ ██╔══██║██╔══██╗██║ ██╔══██║██║ ██║██╔══╝ ██║ ██║██╔══██╗ ██║ ██║██║ ██║╚██████╗██║ ██║██████╔╝███████╗██████╔╝██████╔╝ ╚═╝ ╚═╝╚═╝ ╚═╝ ╚═════╝╚═╝ ╚═╝╚═════╝ ╚══════╝╚═════╝ ╚═════╝ PLAY WITH DATA arcadedb.com

2023-05-03 09:29:20.762 WARNI [ByteArrayOutputStream] ARCADEDB 23.3.1 configuration:

2023-05-03 09:29:20.773 INFO [ArcadeDBServer] ArcadeDB Server v23.3.1 (build 57fcf159173653147b38e84c2ab6904a6de1baa3/1680701788856/main) is starting up... 2023-05-03 09:29:20.785 INFO [ArcadeDBServer] Running on Linux 2.23.432-2343.343.amzn2.x86_64 - OpenJDK 64-Bit Server VM Temurin-11.0.18+10 2023-05-03 09:29:20.785 INFO [ArcadeDBServer] Starting ArcadeDB Server in development mode with plugins [GremlinServer] ... 2023-05-03 09:29:20.801 INFO [ArcadeDBServer] - JMX Metrics Started... 2023-05-03 09:29:20.835 INFO [ServerSecurity] Creating root user with the provided passwordGraph [graph] configured at [./config/gremlin-server.properties] could not be instantiated and will not be available in Gremlin Server. GraphFactory message: GraphFactory could not instantiate this Graph implementation [class com.arcadedb.gremlin.ArcadeGraph] java.lang.RuntimeException: GraphFactory could not instantiate this Graph implementation [class com.arcadedb.gremlin.ArcadeGraph] at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:84) at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:72) at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:106) at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.addGraph(DefaultGraphManager.java:63) at java.base/java.util.LinkedHashMap.forEach(LinkedHashMap.java:684) at org.apache.tinkerpop.gremlin.server.util.DefaultGraphManager.(DefaultGraphManager.java:58) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:490) at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.(ServerGremlinExecutor.java:86) at org.apache.tinkerpop.gremlin.server.GremlinServer.(GremlinServer.java:127) at org.apache.tinkerpop.gremlin.server.GremlinServer.(GremlinServer.java:90) at com.arcadedb.server.gremlin.GremlinServerPlugin.startService(GremlinServerPlugin.java:85) at com.arcadedb.server.ArcadeDBServer.registerPlugins(ArcadeDBServer.java:211) at com.arcadedb.server.ArcadeDBServer.start(ArcadeDBServer.java:141) at com.arcadedb.server.ArcadeDBServer.main(ArcadeDBServer.java:92) Caused by: java.lang.reflect.InvocationTargetException at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:80) ... 16 more Caused by: com.arcadedb.exception.DatabaseOperationException: Cannot create directory './databases/graph' at com.arcadedb.database.EmbeddedDatabase.create(EmbeddedDatabase.java:191) at com.arcadedb.database.DatabaseFactory.create(DatabaseFactory.java:88) at com.arcadedb.gremlin.ArcadeGraph.(ArcadeGraph.java:101) at com.arcadedb.gremlin.ArcadeGraph.open(ArcadeGraph.java:128) ... 21 more

2023-05-03 09:29:21.863 INFO [ServerGremlinExecutor] Initialized Gremlin thread pool. Threads in pool named with pattern gremlin-* 2023-05-03 09:29:21.879 INFO [ServerGremlinExecutor] Initialized GremlinExecutor and preparing GremlinScriptEngines instances.WARNING: An illegal reflective access operation has occurred WARNING: Illegal reflective access by org.codehaus.groovy.reflection.CachedClass (file:/home/arcadedb/lib/arcadedb-gremlin-23.3.1.jar) to method java.lang.Object.finalize() WARNING: Please consider reporting this to the maintainers of org.codehaus.groovy.reflection.CachedClass WARNING: Use --illegal-access=warn to enable warnings of further illegal reflective access operations WARNING: All illegal access operations will be denied in a future release Could not create GremlinScriptEngine for gremlin-groovy java.lang.IllegalStateException: javax.script.ScriptException: javax.script.ScriptException: groovy.lang.MissingPropertyException: No such property: graph for class: Script1 at org.apache.tinkerpop.gremlin.jsr223.DefaultGremlinScriptEngineManager.lambda$createGremlinScriptEngine$16(DefaultGremlinScriptEngineManager.java:464) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183) at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:195) at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) at java.base/java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:274) at java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:484) at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:474) at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:150) at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:173) at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:497) at org.apache.tinkerpop.gremlin.jsr223.DefaultGremlinScriptEngineManager.createGremlinScriptEngine(DefaultGremlinScriptEngineManager.java:450) at org.apache.tinkerpop.gremlin.jsr223.DefaultGremlinScriptEngineManager.getEngineByName(DefaultGremlinScriptEngineManager.java:219) at org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager.lambda$getEngineByName$0(CachedGremlinScriptEngineManager.java:57) at java.base/java.util.concurrent.ConcurrentHashMap.computeIfAbsent(ConcurrentHashMap.java:1705) at org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager.getEngineByName(CachedGremlinScriptEngineManager.java:57) at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:272) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829) Caused by: javax.script.ScriptException: javax.script.ScriptException: groovy.lang.MissingPropertyException: No such property: graph for class: Script1 at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:383) at java.scripting/javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264) at org.apache.tinkerpop.gremlin.jsr223.DefaultGremlinScriptEngineManager.lambda$createGremlinScriptEngine$16(DefaultGremlinScriptEngineManager.java:460) ... 24 more Caused by: javax.script.ScriptException: groovy.lang.MissingPropertyException: No such property: graph for class: Script1 at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:702) at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:381) ... 26 more Caused by: groovy.lang.MissingPropertyException: No such property: graph for class: Script1 at org.codehaus.groovy.runtime.ScriptBytecodeAdapter.unwrap(ScriptBytecodeAdapter.java:65) at org.codehaus.groovy.runtime.callsite.PogoGetPropertySite.getProperty(PogoGetPropertySite.java:51) at org.codehaus.groovy.runtime.callsite.AbstractCallSite.callGroovyObjectGetProperty(AbstractCallSite.java:309) at Script1.run(Script1.groovy:32) at org.apache.tinkerpop.gremlin.groovy.jsr223.GremlinGroovyScriptEngine.eval(GremlinGroovyScriptEngine.java:679) ... 27 more Could not initialize gremlin-groovy GremlinScriptEngine as init script could not be evaluated java.util.concurrent.CompletionException: java.lang.IllegalArgumentException: gremlin-groovy is not an available GremlinScriptEngine at java.base/java.util.concurrent.CompletableFuture.reportJoin(CompletableFuture.java:412) at java.base/java.util.concurrent.CompletableFuture.join(CompletableFuture.java:2044) at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.lambda$new$4(ServerGremlinExecutor.java:153) at java.base/java.util.LinkedHashMap$LinkedKeySet.forEach(LinkedHashMap.java:559) at org.apache.tinkerpop.gremlin.server.util.ServerGremlinExecutor.(ServerGremlinExecutor.java:145) at org.apache.tinkerpop.gremlin.server.GremlinServer.(GremlinServer.java:127) at org.apache.tinkerpop.gremlin.server.GremlinServer.(GremlinServer.java:90) at com.arcadedb.server.gremlin.GremlinServerPlugin.startService(GremlinServerPlugin.java:85) at com.arcadedb.server.ArcadeDBServer.registerPlugins(ArcadeDBServer.java:211) at com.arcadedb.server.ArcadeDBServer.start(ArcadeDBServer.java:141) at com.arcadedb.server.ArcadeDBServer.main(ArcadeDBServer.java:92) Caused by: java.lang.IllegalArgumentException: gremlin-groovy is not an available GremlinScriptEngine at org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager.registerLookUpInfo(CachedGremlinScriptEngineManager.java:95) at org.apache.tinkerpop.gremlin.jsr223.CachedGremlinScriptEngineManager.getEngineByName(CachedGremlinScriptEngineManager.java:58) at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:272) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:829)

2023-05-03 09:29:22.847 INFO [OpLoader] Adding the standard OpProcessor. 2023-05-03 09:29:23.048 INFO [OpLoader] Adding the session OpProcessor. 2023-05-03 09:29:23.050 INFO [OpLoader] Adding the traversal OpProcessor. 2023-05-03 09:29:23.085 INFO [GremlinServer] idleConnectionTimeout was set to 0 which resolves to 0 seconds when configuring this value - this feature will be disabled 2023-05-03 09:29:23.086 INFO [GremlinServer] keepAliveInterval was set to 0 which resolves to 0 seconds when configuring this value - this feature will be disabled 2023-05-03 09:29:23.118 INFO [AbstractChannelizer] Configured application/vnd.graphbinary-v1.0 with org.apache.tinkerpop.gremlin.driver.ser.GraphBinaryMessageSerializerV1 2023-05-03 09:29:23.136 INFO [AbstractChannelizer] Configured application/vnd.gremlin-v3.0+json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0 2023-05-03 09:29:23.136 INFO [AbstractChannelizer] Configured application/json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0 2023-05-03 09:29:23.141 INFO [AbstractChannelizer] Configured application/vnd.gremlin-v2.0+json with org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV2d0 2023-05-03 09:29:23.141 INFO [AbstractChannelizer] application/json already has org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV3d0 configured - it will not be replaced by org.apache.tinkerpop.gremlin.driver.ser.GraphSONMessageSerializerV2d0, change order of serialization configuration if this is not desired. 2023-05-03 09:29:23.187 INFO [ArcadeDBServer] - GremlinServer plugin started 2023-05-03 09:29:23.188 INFO [HttpServer] - Starting HTTP Server (host=0.0.0.0 port=2480-2489 httpsPort=2490-2499)... 2023-05-03 09:29:23.190 INFO [GremlinServer] Gremlin Server configured with worker thread pool of 1, gremlin pool of 16 and boss thread pool of 1. 2023-05-03 09:29:23.190 INFO [GremlinServer] Channel started at port 8182. 2023-05-03 09:29:23.326 INFO [HttpServer] - HTTP Server started (host=0.0.0.0 port=2480 httpsPort=2490) 2023-05-03 09:29:23.339 INFO [ArcadeDBServer] Available query languages: [sqlscript, mongo, gremlin, java, cypher, js, graphql, sql] 2023-05-03 09:29:23.342 INFO [ArcadeDBServer] ArcadeDB Server started in 'development' mode (CPUs=16 MAXRAM=2.00GB) `

Here is my statefulset yaml

`apiVersion: apps/v1 kind: StatefulSet metadata: name: {{ .Release.Name }} labels: app: {{ .Release.Name }} spec: serviceName: {{ .Release.Name }} podManagementPolicy: "OrderedReady" replicas: 1 selector: matchLabels: app: {{ .Release.Name }} template: metadata: labels: app: {{ .Release.Name }} spec: affinity:

Set the anti-affinity selector scope to arcadedb servers.

    podAntiAffinity:
      preferredDuringSchedulingIgnoredDuringExecution:
        - weight: 100
          podAffinityTerm:
            labelSelector:
              matchExpressions:
                - key: app
                  operator: In
                  values:
                    - {{ .Release.Name }}
            topologyKey: kubernetes.io/hostname
  containers:
    - name: {{ .Values.arcadedb.containerName }}
      image: {{ .Values.arcadedb.image }}
      imagePullPolicy: IfNotPresent
      env:
        - name: arcadedb.server.rootPassword
          value: playwithdata
      command:
        - "bin/server.sh"
        - "-Darcadedb.dumpConfigAtStartup=true"
        - "-Darcadedb.server.databaseDirectory=/mnt/data0/databases"
        - "-Darcadedb.server.rootPassword=playwithdata"
        - "-Darcadedb.server.plugins=GremlinServer:com.arcadedb.server.gremlin.GremlinServerPlugin"
      resources:
        requests:
          memory: "512Mi"
      ports:
        - containerPort: 2480
          name: http
        - containerPort: 2424
          name: rpc
        - containerPort: 8182
          name: gremlin-server
      # These volume mounts are persistent. They are like inline claims,
      # but not exactly because the names need to match exactly one of
      # the stateful pod volumes.
      volumeMounts:
        - name: datadir
          mountPath: /mnt/data0
        - name: gremlin-output
          mountPath: /home/arcadedb/databases
  imagePullSecrets:
    - name: regcred

updateStrategy: type: RollingUpdate

These are converted to volume claims by the controller

and mounted at the paths mentioned above.

do not use these in production until ssd GCEPersistentDisk or other ssd pd

volumeClaimTemplates:

humutkazan commented 1 year ago

I changed it like this 

     containers:
        - name: {{ .Values.arcadedb.containerName }}
          image: {{ .Values.arcadedb.image }}
          imagePullPolicy: IfNotPresent
          env:
            - name: arcadedb.server.rootPassword
              value: playwithdata
          command:
            - "bin/server.sh"
            - "-Darcadedb.dumpConfigAtStartup=true"
            - "-Darcadedb.server.databaseDirectory=databases/graph"
            - "-Darcadedb.server.rootPassword=playwithdata"
            - "-Darcadedb.server.plugins=GremlinServer:com.arcadedb.server.gremlin.GremlinServerPlugin"
          resources:
            requests:
              memory: "512Mi"
          ports:
            - containerPort: 2480
              name: http
            - containerPort: 2424
              name: rpc
            - containerPort: 8182
              name: gremlin-server
          # These volume mounts are persistent. They are like inline claims,
          # but not exactly because the names need to match exactly one of
          # the stateful pod volumes.
          volumeMounts:
            - name: datadir
              mountPath: /mnt/data0
            - name: gremlin-output
              mountPath: databases/graph

Edit: there is no error with above configuration but it seems I loose everything under databases/graph directory after pod restart.

lvca commented 1 year ago

Have you tried to use the /mnt/data0 volume for storing the database? Try this:

            - "-Darcadedb.server.databaseDirectory=/mnt/data0/databases"
lvca commented 1 year ago

Any news on this? Closing this issue as resolved. In case please comment here to be reopened.