lealone / Lealone

比 MySQL 和 MongoDB 快10倍的 OLTP 关系数据库和文档数据库
Other
2.44k stars 514 forks source link

Clob数据操作出错,数据无法读取 #195

Closed ContyChen closed 11 months ago

ContyChen commented 1 year ago

clob数据更新操作几次后出错,数据无法读取了 image

lealone 出现异常 image

codefollower commented 1 year ago

发一份测试数据给我调试一下,我的邮箱: codefollower AT gmail.com

codefollower commented 1 year ago

我调试发现 config_info 表的数据跟它的 lob 表的数据不一致了, config_info 表多了5条,content 字段里存放的 lob id 在 lob 表中找不到,所以抛异常。

这个问题跟 https://github.com/lealone/Lealone/issues/192 类似,都是数据不一致引起的。

你自己打包的 lealone-5.2.0-SNAPSHOT.jar 日期是几号的,如果太老打个新的包。 这两个场景我现在重现不了,光是有数据只是知道数据不一致了,但是不知道是哪里的代码存在 bug,单元测试都有索引和 lob 的测试,发现不了这类问题。

下次还遇到这类问题,可以描述一下你当时做了什么操作,这样我可以模拟一下。

codefollower commented 1 year ago

我反反复复更新 config_info 表的 content 字段,都没发现数据不一致,确实挺不好重现。

ContyChen commented 1 year ago

lealone-5.2.0-SNAPSHOT.jar 都是基于最新的代码,github更新了我都会重新打包。 再操作的时候我注意一下,记录一下操作流程,看是否可以重新

ContyChen commented 1 year ago

java进程 内存暴涨,达到了2G多,java.lang.OutOfMemoryError image

lealone日志: image

codefollower commented 1 year ago

可能是 lealone 的 gc 线程没来得急回收 btree page,其他读写线程就把内存撑爆到超过 -Xmx 设置的值了。

codefollower commented 1 year ago

看到你这里举的 lob 相关的例子,我才突然想起来 lealone 的 gc 线程还没有对 lob 表进行 gc,所以 lob 一直读的话就容易 OOM。周末我再把这个问题解决。

ContyChen commented 1 year ago

nacos程序更新数据后会出现大量警告 WARN Async callback is null, may be a bug! packetId = 2275 image

lealone 开启debug日志,total written bytes 信息疯狂刷屏,一直不停。 lealone内存撑爆 image

目前还没有发现触发规律

codefollower commented 1 year ago

代码里执行了哪些 SQL? 出现那个 WARN 基本可以确定执行的某条 SQL 触发了潜在的 bug。

ContyChen commented 1 year ago

跟踪了一下,执行了以下sql, 逻辑:更新 config_info数据 流程:执行INSERT INTO config_info 更新后的新数据,unique key约束冲突,转为update,查询 config_info ,执行 update config_info,upate前的数据插入 his_config_info

2023-07-31 12:06:20,644 INFO [ConfigInfoMapperByLealone] METHOD : insert, SQL : INSERT INTO config_info(data_id, group_id, tenant_id, app_name, content, md5, src_ip, src_user, gmt_create, gmt_modified, c_desc, c_use, effect, type, c_schema, encrypted_data_key) VALUES(?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?), ARGS : [["data_id","group_id","tenant_id","app_name","content","md5","src_ip","src_user","gmt_create","gmt_modified","c_desc","c_use","effect","type","c_schema","encrypted_data_key"]]

2023-07-31 12:06:20,951 INFO [ConfigInfoMapperByLealone] METHOD : select, SQL : SELECT id,data_id,group_id,tenant_id,app_name,content,md5,type,encrypted_data_key FROM config_info WHERE data_id = ? AND group_id = ? AND tenant_id = ?, ARGS : [["id","data_id","group_id","tenant_id","app_name","content","md5","type","encrypted_data_key"],["data_id","group_id","tenant_id"]]

2023-07-31 12:06:20,974 INFO [ConfigInfoMapperByLealone] METHOD : update, SQL : UPDATE config_info SET content = ?,md5 = ?,src_ip = ?,src_user = ?,gmt_modified = ?,app_name = ?,c_desc = ?,c_use = ?,effect = ?,type = ?,c_schema = ?,encrypted_data_key = ? WHERE data_id = ? AND group_id = ? AND tenant_id = ?, ARGS : [["content","md5","src_ip","src_user","gmt_modified","app_name","c_desc","c_use","effect","type","c_schema","encrypted_data_key"],["data_id","group_id","tenant_id"]]

2023-07-31 12:06:21,005 INFO [HistoryConfigInfoMapperByLealone] METHOD : insert, SQL : INSERT INTO his_config_info(id, data_id, group_id, tenant_id, app_name, content, md5, src_ip, src_user, gmt_modified, op_type, encrypted_data_key) VALUES(?,?,?,?,?,?,?,?,?,?,?,?), ARGS : [["id","data_id","group_id","tenant_id","app_name","content","md5","src_ip","src_user","gmt_modified","op_type","encrypted_data_key"]]

2023-07-31 12:06:21,113 INFO [ConfigTagsRelationMapperByLealone] METHOD : select, SQL : SELECT tag_name FROM config_tags_relation WHERE data_id = ? AND group_id = ? AND tenant_id = ?, ARGS : [["tag_name"],["data_id","group_id","tenant_id"]]

2023-07-31 12:06:21,135 INFO [ConfigInfoMapperByLealone] METHOD : select, SQL : SELECT id,data_id,group_id,tenant_id,app_name,content,md5,gmt_create,gmt_modified,src_user,src_ip,c_desc,c_use,effect,type,c_schema,encrypted_data_key FROM config_info WHERE data_id = ? AND group_id = ? AND tenant_id = ?, ARGS : [["id","data_id","group_id","tenant_id","app_name","content","md5","gmt_create","gmt_modified","src_user","src_ip","c_desc","c_use","effect","type","c_schema","encrypted_data_key"],["data_id","group_id","tenant_id"]]
codefollower commented 1 year ago

以上语句没有什么特殊的,我在你给我的测试数据上面测,将它们单独跑或者放到一个事务中跑都没有抛出异常。

你在 org.lealone.server.Scheduler 类的第241行代码打个断点一步步调试一下。

执行完第241行代码时 currentSession.yieldableCommand 都是 null,如果不是 null 大概率就是个 bug。

太奇怪了,你报告的3个问题我一个都没法重现。

codefollower commented 1 year ago

lealone-5.2.0-SNAPSHOT.zip 这是我打的 jar 包,可以试试。