search

apache / incubator-hugegraph

A graph database that supports more than 100+ billion data, high performance and scalability (Include OLTP Engine & REST-API & Backends)
https://hugegraph.apache.org
Apache License 2.0
2.58k stars 512 forks source link

[Bug] Dynamic create multi graph error with restful api in 1.2.0 #2418

Closed fantengfeiNana closed 4 months ago

fantengfeiNana commented 5 months ago

Problem Type (问题类型)

other exception / error (其他异常报错)

Before submit

Environment (环境信息)

Your Question (问题描述)

我们使用了下载二进制压缩包的方式完成了 apache-hugegraph-incubating-1.2.0的下载和部署， 已经完成了初始化和hugegraph-server的启动，获取初始化的接口调用正常。 但是在通过restful接口创建图谱时报错，通过查看hugegraph-server.log日志发现是数据库连接异常 如下图： image 能够确定是的已经在hugegraph.properties配置文件按照文档完成了存储配置

Vertex/Edge example (问题点 / 边数据举例)

No response

Schema [VertexLabel, EdgeLabel, IndexLabel] (元数据结构)

No response

dosubot[bot] commented 5 months ago

🤖

你好 @fantengfeiNana，

我是Dosu，一个机器人，我可以帮助你解决bug，回答问题，甚至成为一个贡献者。请知道，在等待人类维护者的同时，我在这里帮助你。我正在查看你的问题，会尽快给你一个完整的答案。

谢谢你的耐心等待！

fantengfeiNana commented 5 months ago

我本地通过idea调试源码是通过手动修改MysqlOptions.java文件的中数据库配置解决的这个问题。通过二进制文件部署，请问我该如何解决这个问题？

imbajin commented 5 months ago

我本地通过idea调试源码是通过手动修改MysqlOptions.java文件的中数据库配置解决的这个问题。通过二进制文件部署，请问我该如何解决这个问题？

Known Issue in 1.2.0, we will fix it soon (follow our next PR changes maybe)

Currently, you can temporarily use your current way(hard code), or temporarily roll back 1.0.0 and wait for the community to fix and update. We will also release a new fix release for 1.2.1

JackyYangPassion commented 5 months ago

According to the official documentation, the BUG has not been reproduced.

First Manually download mysql-diver and place it in the lib directory. mysql-connector-java-8.0.30.jar

Second Dynamically create a graph using the API:

POST http://127.0.0.1:8081/graphs/test_mysql

Body:

   gremlin.graph=org.apache.hugegraph.HugeFactory
   backend=mysql
   serializer=mysql
   store=test_mysql
   jdbc.driver=com.mysql.jdbc.Driver
   jdbc.url=jdbc:mysql://127.0.0.1:3306
   jdbc.username=root
   jdbc.password=123456
   jdbc.reconnect_max_times=3
   jdbc.reconnect_interval=3
   jdbc.sslmode=false

Please provide the detailed error log and the configuration content for creating the graph in the body to assist with troubleshooting. @fantengfeiNana

simon824 commented 5 months ago

@JackyYangPassion @imbajin Dynamically creating graph on rocksdb backend also failed, but it seems a different exception

backend=rocksdb serializer=binary store=test


- Server exception
```java
2024-01-31 01:44:28 [db-open-1] [ERROR] o.a.h.b.s.r.RocksDBStore - Failed to open RocksDB 'rocksdb-data/m'
org.rocksdb.RocksDBException: lock hold by current process, acquire time 1704272264 acquiring thread 139698267633408: rocksdb-data/m/LOCK: No locks available
        at org.rocksdb.RocksDB.open(Native Method) ~[rocksdbjni-7.2.2.jar:?]
        at org.rocksdb.RocksDB.open(RocksDB.java:305) ~[rocksdbjni-7.2.2.jar:?]
        at org.apache.hugegraph.backend.store.rocksdb.RocksDBStdSessions.openRocksDB(RocksDBStdSessions.java:413) ~[hugegraph-rocksdb-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.backend.store.rocksdb.RocksDBStdSessions.<init>(RocksDBStdSessions.java:102) ~[hugegraph-rocksdb-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.backend.store.rocksdb.RocksDBStore.openSessionPool(RocksDBStore.java:379) ~[hugegraph-rocksdb-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.backend.store.rocksdb.RocksDBStore.open(RocksDBStore.java:312) ~[hugegraph-rocksdb-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.backend.store.rocksdb.RocksDBStore.open(RocksDBStore.java:304) ~[hugegraph-rocksdb-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.backend.store.rocksdb.RocksDBStore.lambda$open$3(RocksDBStore.java:227) ~[hugegraph-rocksdb-1.2.0.jar:1.2.0]
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) ~[?:?]
        at java.util.concurrent.FutureTask.run(FutureTask.java:264) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
        at java.lang.Thread.run(Thread.java:829) [?:?]
2024-01-31 01:44:28 [grizzly-http-server-13] [ERROR] o.a.h.c.GraphManager - Failed to create graph 'test' due to: GraphFactory could not instantiate this Graph implementation [class org.apache.hugegraph.HugeFactory]
java.lang.RuntimeException: GraphFactory could not instantiate this Graph implementation [class org.apache.hugegraph.HugeFactory]
        at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:84) ~[gremlin-core-3.5.1.jar:3.5.1]
        at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:72) ~[gremlin-core-3.5.1.jar:3.5.1]
        at org.apache.hugegraph.core.GraphManager.createGraph(GraphManager.java:583) ~[hugegraph-api-1.2.0.jar:0.71.0.0]
        at org.apache.hugegraph.core.GraphManager.createGraph(GraphManager.java:186) ~[hugegraph-api-1.2.0.jar:0.71.0.0]
        at org.apache.hugegraph.api.profile.GraphsAPI.create(GraphsAPI.java:136) ~[hugegraph-api-1.2.0.jar:0.71.0.0]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory.lambda$static$0(ResourceMethodInvocationHandlerFactory.java:52) ~[jersey-server-3.0.3.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:124) ~[jersey-server-3.0.3.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:167) ~[jersey-server-3.0.3.jar:?]
        at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$ObjectOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:192) ~[jersey-server-3.0.3.jar:?]
        at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:79) ~[jersey-server-3.0.3.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:475) ~[jersey-server-3.0.3.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:397) ~[jersey-server-3.0.3.jar:?]
        at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:81) ~[jersey-server-3.0.3.jar:?]
        at org.glassfish.jersey.server.ServerRuntime$1.run(ServerRuntime.java:255) ~[jersey-server-3.0.3.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:248) ~[jersey-common-3.0.3.jar:?]
        at org.glassfish.jersey.internal.Errors$1.call(Errors.java:244) ~[jersey-common-3.0.3.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:292) ~[jersey-common-3.0.3.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:274) ~[jersey-common-3.0.3.jar:?]
        at org.glassfish.jersey.internal.Errors.process(Errors.java:244) ~[jersey-common-3.0.3.jar:?]
        at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:265) ~[jersey-common-3.0.3.jar:?]
        at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:234) ~[jersey-server-3.0.3.jar:?]
        at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:684) ~[jersey-server-3.0.3.jar:?]
        at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:356) ~[jersey-container-grizzly2-http-3.0.3.jar:?]
        at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:190) ~[grizzly-http-server-3.0.1.jar:3.0.1]
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:535) ~[grizzly-framework-3.0.1.jar:3.0.1]
        at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:515) ~[grizzly-framework-3.0.1.jar:3.0.1]
        at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: java.lang.reflect.InvocationTargetException
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:80) ~[gremlin-core-3.5.1.jar:3.5.1]
        ... 30 more
Caused by: org.apache.hugegraph.exception.ConnectionException: Failed to open RocksDB store
        at org.apache.hugegraph.backend.store.rocksdb.RocksDBStore.waitOpenFinished(RocksDBStore.java:261) ~[hugegraph-rocksdb-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.backend.store.rocksdb.RocksDBStore.open(RocksDBStore.java:249) ~[hugegraph-rocksdb-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.backend.tx.AbstractTransaction.<init>(AbstractTransaction.java:80) ~[hugegraph-core-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.backend.tx.IndexableTransaction.<init>(IndexableTransaction.java:28) ~[hugegraph-core-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.backend.tx.SchemaTransaction.<init>(SchemaTransaction.java:79) ~[hugegraph-core-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.backend.cache.CachedSchemaTransaction.<init>(CachedSchemaTransaction.java:53) ~[hugegraph-core-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.StandardHugeGraph.openSchemaTransaction(StandardHugeGraph.java:460) ~[hugegraph-core-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.StandardHugeGraph.access$2700(StandardHugeGraph.java:121) ~[hugegraph-core-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.StandardHugeGraph$TinkerPopTransaction.getOrNewTransaction(StandardHugeGraph.java:1471) ~[hugegraph-core-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.StandardHugeGraph$TinkerPopTransaction.doOpen(StandardHugeGraph.java:1380) ~[hugegraph-core-1.2.0.jar:1.2.0]
        at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.open(AbstractTransaction.java:95) ~[gremlin-core-3.5.1.jar:3.5.1]
        at org.apache.tinkerpop.gremlin.structure.Transaction$READ_WRITE_BEHAVIOR$1.accept(Transaction.java:213) ~[gremlin-core-3.5.1.jar:3.5.1]
        at org.apache.tinkerpop.gremlin.structure.Transaction$READ_WRITE_BEHAVIOR$1.accept(Transaction.java:210) ~[gremlin-core-3.5.1.jar:3.5.1]
        at org.apache.tinkerpop.gremlin.structure.util.AbstractThreadLocalTransaction.doReadWrite(AbstractThreadLocalTransaction.java:92) ~[gremlin-core-3.5.1.jar:3.5.1]
        at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.readWrite(AbstractTransaction.java:136) ~[gremlin-core-3.5.1.jar:3.5.1]
        at org.apache.hugegraph.StandardHugeGraph.graphTransaction(StandardHugeGraph.java:534) ~[hugegraph-core-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.StandardHugeGraph.backendStoreFeatures(StandardHugeGraph.java:266) ~[hugegraph-core-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.StandardHugeGraph.<init>(StandardHugeGraph.java:226) ~[hugegraph-core-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.HugeFactory.open(HugeFactory.java:88) ~[hugegraph-core-1.2.0.jar:1.2.0]
        at org.apache.hugegraph.HugeFactory.open(HugeFactory.java:66) ~[hugegraph-core-1.2.0.jar:1.2.0]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:?]
        at jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:?]
        at jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:?]
        at java.lang.reflect.Method.invoke(Method.java:566) ~[?:?]
        at org.apache.tinkerpop.gremlin.structure.util.GraphFactory.open(GraphFactory.java:80) ~[gremlin-core-3.5.1.jar:3.5.1]
        ... 30 more
JackyYangPassion commented 5 months ago

@simon824 use config with postman Dynamically creating graph on rocksdb backend success!

image

success log

2024-02-01 22:54:12 [db-open-1] [INFO] o.a.h.b.s.r.RocksDBStore - Failed to open RocksDB 'rocksdb-data/m' with database 'test_rockedb', try to init CF later
2024-02-01 22:54:12 [grizzly-http-server-4] [INFO] o.a.h.b.c.CacheManager - Init RamCache for 'schema-id-test_rockedb' with capacity 10000
2024-02-01 22:54:12 [grizzly-http-server-4] [INFO] o.a.h.b.c.CacheManager - Init RamCache for 'schema-name-test_rockedb' with capacity 10000
2024-02-01 22:54:12 [db-open-1] [INFO] o.a.h.b.s.r.RocksDBStore - Opening RocksDB with data path: rocksdb-data/s
2024-02-01 22:54:12 [db-open-1] [INFO] o.a.h.b.s.r.RocksDBStore - Opening RocksDB with data path: rocksdb-data/g
2024-02-01 22:54:12 [grizzly-http-server-4] [INFO] o.a.h.b.c.CacheManager - Init LevelCache for 'vertex-test_rockedb' with capacity 10000:10000000
2024-02-01 22:54:12 [grizzly-http-server-4] [INFO] o.a.h.b.c.CacheManager - Init LevelCache for 'edge-test_rockedb' with capacity 1000:1000000
2024-02-01 22:54:12 [grizzly-http-server-4] [INFO] o.a.h.b.c.CacheManager - Init RamCache for 'users-test_rockedb' with capacity 10240
2024-02-01 22:54:12 [grizzly-http-server-4] [INFO] o.a.h.b.c.CacheManager - Init RamCache for 'users_pwd-test_rockedb' with capacity 10240
2024-02-01 22:54:12 [grizzly-http-server-4] [INFO] o.a.h.b.c.CacheManager - Init RamCache for 'token-test_rockedb' with capacity 10240
2024-02-01 22:54:12 [grizzly-http-server-4] [INFO] o.a.h.b.s.r.RocksDBStore - Write down the backend version: 1.11
2024-02-01 22:54:12 [grizzly-http-server-4] [INFO] o.a.h.StandardHugeGraph - Graph 'test_rockedb' has been initialized
2024-02-01 22:54:12 [grizzly-http-server-4] [INFO] o.a.h.StandardHugeGraph - Init system info for graph 'test_rockedb'
2024-02-01 22:54:12 [grizzly-http-server-4] [INFO] o.a.h.StandardHugeGraph - Init server info [server-1-MASTER] for graph 'test_rockedb'...

my local Environment (环境信息) Server Version: 1.2.0 (Apache Release Version) java version: jdk 11

JackyYangPassion commented 4 months ago

Log

2024-01-31 01:44:28 [db-open-1] [ERROR] o.a.h.b.s.r.RocksDBStore - Failed to open RocksDB 'rocksdb-data/m'
org.rocksdb.RocksDBException: lock hold by current process, acquire time 1704272264 acquiring thread 139698267633408: rocksdb-data/m/LOCK: No locks available

Root cause: In Docker, the default directory has already started a RocksDB instance, so when the data and WAL directories are not specified, it can lead to a RocksDB lock conflict.

When launching Docker, the normal process for dynamically creating graph configurations includes adding a data directory. The specific configuration is as follows:

gremlin.graph=org.apache.hugegraph.HugeFactory
vertex.cache_type=l2
edge.cache_type=l2
backend=rocksdb
serializer=binary
store=test
rocksdb.data_path=/hugegraph-server/data/test/data
rocksdb.wal_path=/hugegraph-server/data/test/wal
JackyYangPassion commented 4 months ago

The root cause is not an error in the code logic; it is necessary to configure the data directory according to the documentation. doc