search

apache / incubator-hugegraph

A graph database that supports more than 100+ billion data, high performance and scalability (Include OLTP Engine & REST-API & Backends)
https://hugegraph.apache.org
Apache License 2.0
2.62k stars 518 forks source link

[Question] hugegraph 0.12.0 后端hbase使用 #1730

Closed chencjiajy closed 2 years ago

chencjiajy commented 2 years ago

Problem Type (问题类型)

rest-api (结果不合预期)

Before submit

Environment (环境信息)

Your Question (问题描述)

  1. 安装hugegraph 0.12.0 配置图时，如果写serializer=hbase， 初始化时会失败，报错“com.baidu.hugegraph.backend.BackendException: Not exists serializer: 'hhase'” 如果写serializer=binary， 可以成功初始化图， 所以现在的hugegraph 0.12.0的serializer为binary.

  2. 使用hugegraph tools 1.6.0 将数据从hugegraph 0.11.2（serializer为hbase) 迁移到 hugegraph 0.12.0。

    • 备份 ./bin/hugegraph --url http://hugegraph-server-0112:8183 --graph test6 backup -t all --directory ./backup-test
    • restore到0.12.0 : 设置图模式 ./bin/hugegraph --url http://hugegraph-server-0120:8183 --graph test10 graph-mode-set -m RESTORING 执行恢复操作 ./bin/hugegraph --url http://hugegraph-server-0120:8183 --graph test10 restore -t all --directory ./backup-test

  3. 然后再使用api 或gremlin查询数据，只查询节点没有问题，如果查询节点的邻居，会报错

    2022-01-12 15:00:50 [grizzly-http-server-5] [ERROR] c.b.h.b.Transaction - Failed to parse entry: 0x9231363a6c706c756e33316d6f3178616f6e777a: [?17:Mute!Boolean!lplun31mo1xaonwz!10=, ?17:Direction!Enum!lplun31mo1xaonwz!10=, ?17:Switch!Boolean!lplun31mo1xaonwz!10=, ?17:Volume!Integer!lplun31mo1xaonwz!10=, ?17:Channel!Integer!lplun31mo1xaonwz!10=] java.lang.IllegalStateException: Invalid entry) with unknown type(162): 0x82081900a231373a4d75746521426f6f6c65616e216c706c756e33316d6f3178616f6e777a213130 at com.google.common.base.Preconditions.checkState(Preconditions.java:531) ~[guava-25.1-jre.jar:?] at com.baidu.hugegraph.util.E.checkState(E.java:68) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.backend.serializer.BinarySerializer.parseColumn(BinarySerializer.java:324) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.backend.serializer.BinarySerializer.readVertex(BinarySerializer.java:454) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.backend.tx.GraphTransaction.parseEntry(GraphTransaction.java:1882) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.backend.tx.GraphTransaction.lambda$queryEdgesFromBackend$7(GraphTransaction.java:986) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.iterator.FlatMapperIterator.fetch(FlatMapperIterator.java:62) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.iterator.WrappedIterator.hasNext(WrappedIterator.java:40) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.backend.cache.CachedGraphTransaction.queryEdgesFromBackend(CachedGraphTransaction.java:310) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.backend.tx.GraphTransaction.queryEdges(GraphTransaction.java:944) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.StandardHugeGraph.edges(StandardHugeGraph.java:670) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.auth.HugeGraphAuthProxy.edges(HugeGraphAuthProxy.java:488) ~[hugegraph-api-0.12.0.jar:0.67.0.0] at com.baidu.hugegraph.traversal.algorithm.HugeTraverser.edgesOfVertex(HugeTraverser.java:174) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.traversal.algorithm.HugeTraverser.adjacentVertices(HugeTraverser.java:124) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.traversal.algorithm.KneighborTraverser.kneighbor(KneighborTraverser.java:61) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.api.traversers.KneighborAPI.get(KneighborAPI.java:96) ~[hugegraph-api-0.12.0.jar:0.67.0.0] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[?:1.8.0_231] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[?:1.8.0_231] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[?:1.8.0_231] at java.lang.reflect.Method.invoke(Method.java:498) ~[?:1.8.0_231] at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81) ~[jersey-server-2.25.1.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144) ~[jersey-server-2.25.1.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161) ~[jersey-server-2.25.1.jar:?] at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205) ~[jersey-server-2.25.1.jar:?] at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99) ~[jersey-server-2.25.1.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389) ~[jersey-server-2.25.1.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347) ~[jersey-server-2.25.1.jar:?] at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102) ~[jersey-server-2.25.1.jar:?] at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326) ~[jersey-server-2.25.1.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271) ~[jersey-common-2.25.1.jar:?] at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267) ~[jersey-common-2.25.1.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:315) ~[jersey-common-2.25.1.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:297) ~[jersey-common-2.25.1.jar:?] at org.glassfish.jersey.internal.Errors.process(Errors.java:267) ~[jersey-common-2.25.1.jar:?] at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317) ~[jersey-common-2.25.1.jar:?] at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305) ~[jersey-server-2.25.1.jar:?] at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154) ~[jersey-server-2.25.1.jar:?] at org.glassfish.jersey.grizzly2.httpserver.GrizzlyHttpContainer.service(GrizzlyHttpContainer.java:384) ~[jersey-container-grizzly2-http-2.25.1.jar:?] at org.glassfish.grizzly.http.server.HttpHandler$1.run(HttpHandler.java:200) ~[grizzly-http-server-2.4.4.jar:2.4.4] at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:569) ~[grizzly-framework-2.4.4.jar:2.4.4] at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:549) ~[grizzly-framework-2.4.4.jar:2.4.4] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_231]

类似的issue: https://github.com/hugegraph/hugegraph/issues/1590 , https://github.com/hugegraph/hugegraph/issues/193

Vertex/Edge example (问题点 / 边数据举例)

No response

Schema [VertexLabel, EdgeLabel, IndexLabel] (元数据结构)

No response

imbajin commented 2 years ago

安装 hugegraph 0.12.0 配置图时，如果写 serializer=hbase， 初始化时会失败，报错 “com.baidu.hugegraph.backend.BackendException: Not exists serializer: 'hhase'” 如果写 serializer=binary， 可以成功初始化图， 所以现在的 hugegraph 0.12.0 的 serializer 为 binary.

从报错看是因为你写错了把, 你把 hbase 写成了 hhase?

第二个 hbase 的写入问题可以复现么? 最简单的办法, 先不用 tools 导入, 先初始化一个空的 0.12 的 hbase 后端, 然后你手动通过 rest-api 插入一个点, 然后它的两条边查询试试, 如果可以复现麻烦把报错信息贴一下

chencjiajy commented 2 years ago

安装 hugegraph 0.12.0 配置图时，如果写 serializer=hbase， 初始化时会失败，报错 “com.baidu.hugegraph.backend.BackendException: Not exists serializer: 'hhase'” 如果写 serializer=binary， 可以成功初始化图， 所以现在的 hugegraph 0.12.0 的 serializer 为 binary.

从报错看是因为你写错了把, 你把 hbase 写成了 hhase?

第二个 hbase 的写入问题可以复现么? 最简单的办法, 先不用 tools 导入, 先初始化一个空的 0.12 的 hbase 后端, 然后你手动通过 rest-api 插入一个点, 然后它的两条边查询试试, 如果可以复现麻烦把报错信息贴一下

@imbajin 当serializer为binary的时候，使用了一个空的图，用rest-api插入节点和边之后，查询边也会报错。 尝试了我们自己的数据以及client例子中数据，都会报错。

2022-01-19 10:39:58 [gremlin-server-exec-3] [ERROR] c.b.h.b.Transaction - Failed to parse entry: 0x86313a6d61726b6f: [?1:josh=??????, ?1:vadas=???????, ?2:lop=???????] java.lang.IllegalStateException: Invalid entry) with unknown type(133): 0x8208010085313a6a6f7368 at com.google.common.base.Preconditions.checkState(Preconditions.java:531) ~[guava-25.1-jre.jar:?] at com.baidu.hugegraph.util.E.checkState(E.java:68) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.backend.serializer.BinarySerializer.parseColumn(BinarySerializer.java:324) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.backend.serializer.BinarySerializer.readVertex(BinarySerializer.java:454) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.backend.tx.GraphTransaction.parseEntry(GraphTransaction.java:1882) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.backend.tx.GraphTransaction.lambda$queryEdgesFromBackend$7(GraphTransaction.java:986) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.iterator.FlatMapperIterator.fetch(FlatMapperIterator.java:62) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.iterator.WrappedIterator.hasNext(WrappedIterator.java:40) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.iterator.FilterIterator.fetch(FilterIterator.java:42) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.iterator.WrappedIterator.hasNext(WrappedIterator.java:40) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.iterator.LimitIterator.fetch(LimitIterator.java:42) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.iterator.WrappedIterator.hasNext(WrappedIterator.java:40) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.iterator.FilterIterator.fetch(FilterIterator.java:42) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.iterator.WrappedIterator.hasNext(WrappedIterator.java:40) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep.processNextStart(GraphStep.java:147) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep.processNextStart(FilterStep.java:37) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:197) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils.fill(IteratorUtils.java:62) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils.list(IteratorUtils.java:85) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils.asList(IteratorUtils.java:382) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler.lambda$channelRead$1(HttpGremlinEndpointHandler.java:245) ~[gremlin-server-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.util.function.FunctionUtils.lambda$wrapFunction$0(FunctionUtils.java:36) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:272) ~[gremlin-groovy-3.4.3.jar:3.4.3] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_231] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_231] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_231] at com.baidu.hugegraph.auth.HugeGraphAuthProxy$ContextTask.run(HugeGraphAuthProxy.java:1748) ~[hugegraph-api-0.12.0.jar:0.67.0.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_231] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_231] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_231] 2022-01-19 10:39:58 [gremlin-server-exec-3] [ERROR] c.b.h.b.Transaction - Failed to parse entry: 0x86313a7065746572: [?2:lop=???????] java.lang.IllegalStateException: Invalid entry) with unknown type(132): 0x8208020084323a6c6f70 at com.google.common.base.Preconditions.checkState(Preconditions.java:531) ~[guava-25.1-jre.jar:?] at com.baidu.hugegraph.util.E.checkState(E.java:68) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.backend.serializer.BinarySerializer.parseColumn(BinarySerializer.java:324) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.backend.serializer.BinarySerializer.readVertex(BinarySerializer.java:454) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.backend.tx.GraphTransaction.parseEntry(GraphTransaction.java:1882) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.backend.tx.GraphTransaction.lambda$queryEdgesFromBackend$7(GraphTransaction.java:986) ~[hugegraph-core-0.12.0.jar:0.12.0.0] at com.baidu.hugegraph.iterator.FlatMapperIterator.fetch(FlatMapperIterator.java:62) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.iterator.WrappedIterator.hasNext(WrappedIterator.java:40) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.iterator.FilterIterator.fetch(FilterIterator.java:42) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.iterator.WrappedIterator.hasNext(WrappedIterator.java:40) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.iterator.LimitIterator.fetch(LimitIterator.java:42) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.iterator.WrappedIterator.hasNext(WrappedIterator.java:40) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.iterator.FilterIterator.fetch(FilterIterator.java:42) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at com.baidu.hugegraph.iterator.WrappedIterator.hasNext(WrappedIterator.java:40) ~[hugegraph-common-2.0.0.jar:2.0.0.0] at org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep.processNextStart(GraphStep.java:147) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep.processNextStart(FilterStep.java:37) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.process.traversal.util.DefaultTraversal.hasNext(DefaultTraversal.java:197) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils.fill(IteratorUtils.java:62) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils.list(IteratorUtils.java:85) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.util.iterator.IteratorUtils.asList(IteratorUtils.java:382) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.server.handler.HttpGremlinEndpointHandler.lambda$channelRead$1(HttpGremlinEndpointHandler.java:245) ~[gremlin-server-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.util.function.FunctionUtils.lambda$wrapFunction$0(FunctionUtils.java:36) ~[gremlin-core-3.4.3.jar:3.4.3] at org.apache.tinkerpop.gremlin.groovy.engine.GremlinExecutor.lambda$eval$0(GremlinExecutor.java:272) ~[gremlin-groovy-3.4.3.jar:3.4.3] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_231] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) ~[?:1.8.0_231] at java.util.concurrent.FutureTask.run(FutureTask.java:266) ~[?:1.8.0_231] at com.baidu.hugegraph.auth.HugeGraphAuthProxy$ContextTask.run(HugeGraphAuthProxy.java:1748) ~[hugegraph-api-0.12.0.jar:0.67.0.0] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) ~[?:1.8.0_231] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) ~[?:1.8.0_231] at java.lang.Thread.run(Thread.java:748) [?:1.8.0_231]

感觉应该是serializer只能使用hbase吧， 我之前确实没有发现我把hbase拼错了，从报错没看出来😂