apache / incubator-hugegraph

A graph database that supports more than 100+ billion data, high performance and scalability (Include OLTP Engine & REST-API & Backends)
https://hugegraph.apache.org
Apache License 2.0
2.65k stars 518 forks source link

[Bug] mysql 后端运行一段时间可能内存泄漏 #1626

Closed z7658329 closed 3 years ago

z7658329 commented 3 years ago

Bug Type (问题类型)

No response

Before submit

Environment (环境信息)

Expected & Actual behavior (期望与实际表现)

1.参数配置说明:

  1. backend后端mysql 实际数据量 顶点数 50W 边数量 130W
  2. 有个业务进程会对huge CRUD,大概的读写 每分钟2000左右 大部分是read
  3. jvm配置: java -server -Dname=HugeGraphServer -Dlog4j.configurationFile=/scripts/hugegraph-0.11.2/conf/log4j2.xml -Djava.security.manager=com.baidu.hugegraph.security.HugeSecurityManager -Xms16g -Xmx16g -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/scripts/hugegraph-0.11.2/logs -XX:+UseGCLogFileRotation -XX:GCLogFileSize=10M -XX:NumberOfGCLogFiles=3 -Xloggc:/scripts/hugegraph-0.11.2/logs/gc.log -XX:+PrintHeapAtGC -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+UseG1GC -XX:MaxGCPauseMillis=1000 -XX:+ParallelRefProcEnabled -XX:InitiatingHeapOccupancyPercent=50 -XX:G1RSetUpdatingPauseTimePercent=5 4.顶点边的缓存配置: hugegraph.properties:开启了l2 堆外 vertex.cache_type=l2 vertex.cache_expire=300 edge.cache_type=l2 edge.cache_expire=300

    2.期望表现

    1.在发生老年代 GC 时,可以回收掉一部分对象,使得老年代使用量不是持续走高。(实际运行一段时间后高居不下最后FGC,时长高达43s)

3.实际情况

gc

jmap统计对象数据: image 存活对象: f0abcbe2ba279b0fb74316a9ba99ab0

GC日志: gc.log.zip

Vertex/Edge example (问题点 / 边数据举例)

No response

Schema [VertexLabel, EdgeLabel, IndexLabel] (元数据结构)

No response

javeme commented 3 years ago

分析了一下,是 MySQL Statement 未及时关闭导致的,可以按照如下方式修复:

image

修改为如下步骤:

Statement statement = this.conn.createStatement();
ResultSet rs = statement.executeQuery(sql);
return new ResultSetWrapper(rs, statement);

然后通过ResultSetWrapper的close()方法来关闭ResultSet和Statement,通过results()方法来获得实际的ResultSet:

class ResultSetWrapper {
    ResultSet rs;
    Statement statement;

    ResultSet results() {
        return this.rs;
    }

    close() {
        this.rs.close();
        this.statement.close();
    }
}

历史用户反馈类似问题:https://github.com/hugegraph/hugegraph/issues/1203#issuecomment-720980290

z7658329 commented 2 years ago

count语句未关闭statement

b4102f88928667f1f5c1e42f78f3eb3