search

apache / incubator-hugegraph

A graph database that supports more than 100+ billion data, high performance and scalability (Include OLTP Engine & REST-API & Backends)
https://hugegraph.apache.org
Apache License 2.0
2.62k stars 518 forks source link

请问，server 入库，有遇到 OOM 异常吗？？ #451

Closed ChenBinaBGitHub closed 2 years ago

ChenBinaBGitHub commented 5 years ago

server 入库 时，大约有1亿条边，入库至4000w时，就会抛出OOM，异常，如下：

Exception in thread "main" java.lang.OutOfMemoryError: GC overhead limit exceeded at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57) at java.nio.ByteBuffer.allocate(ByteBuffer.java:335) at com.baidu.hugegraph.backend.serializer.BytesBuffer.(BytesBuffer.java:67) at com.baidu.hugegraph.backend.serializer.BytesBuffer.allocate(BytesBuffer.java:76) at com.baidu.hugegraph.backend.serializer.BinarySerializer.formatEdgeName(BinarySerializer.java:174) at com.baidu.hugegraph.backend.serializer.BinarySerializer.newBackendEntry(BinarySerializer.java:88) at com.baidu.hugegraph.backend.serializer.BinarySerializer.writeEdge(BinarySerializer.java:382) at com.baidu.hugegraph.backend.tx.GraphTransaction.prepareAdditions(GraphTransaction.java:238) at com.baidu.hugegraph.backend.tx.GraphTransaction.prepareCommit(GraphTransaction.java:208) at com.baidu.hugegraph.backend.tx.IndexableTransaction.commit2Backend(IndexableTransaction.java:53) at com.baidu.hugegraph.backend.tx.AbstractTransaction.commit(AbstractTransaction.java:166) at com.baidu.hugegraph.backend.tx.GraphTransaction.commit(GraphTransaction.java:289) at com.baidu.hugegraph.HugeGraph$TinkerpopTransaction.doCommit(HugeGraph.java:650) at org.apache.tinkerpop.gremlin.structure.util.AbstractTransaction.commit(AbstractTransaction.java:105) at com.baidu.hugegraph.HugeGraph$TinkerpopTransaction.commit(HugeGraph.java:612) at com.suntek.ai.graphdb.data.FactoryLoadToGraph$$anonfun$loadEdge$1.apply(FactoryLoadToGraph.scala:111) at com.suntek.ai.graphdb.data.FactoryLoadToGraph$$anonfun$loadEdge$1.apply(FactoryLoadToGraph.scala:103) at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33) at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)

javeme commented 5 years ago

@ChenBinaBGitHub 请提供下环境信息（空闲内存、JVM分配内存、后端存储类型）。插入时OOM一般是内存不足导致的。

ChenBinaBGitHub commented 5 years ago

空闲内存：30g,JVM分配内存40g, 后端存储类型：HBase 初步发现问题是在入边时 对于事务性提交， graph.close() 后，graph 再次open 时 进行 src,addEdge(label,dst) 时， 就会报 graph 没有打开，Graph hugeGraph has been closed

换句话说，再入边时，无法对graph 进行关闭再打开操作(事务性操作)

github-actions[bot] commented 2 years ago

Due to the lack of activity, the current issue is marked as stale and will be closed after 20 days, any update will remove the stale label