armink / FlashDB

An ultra-lightweight database that supports key-value and time series data | 一款支持 KV 数据和时序数据的超轻量级数据库
Apache License 2.0
1.9k stars 433 forks source link

在FLASHDB 中建立多分区多DB运行出现问题 #325

Open tianzhilianwo opened 1 week ago

tianzhilianwo commented 1 week ago

朱工:您好!再用FLASH DB做测试时,发现一下问题,通过FAL建立多分多数据库时,KV出现不GC的情况,跟踪了一下,在最后一个GC时 会改变part_falsh_cache的初始化值,导致无法寻找到对应的flashdev 但是,采用两个分区时可以正常运行的;请教一下是什么问题?如何解决?

define FAL_PART_TABLE \

{ \ {FAL_PART_MAGIC_WROD, "fdb_tsdb0", "norflash0", 0L, 2L1024L1024L, 0}, \ {FAL_PART_MAGIC_WROD, "fdb_tsdb1", "norflash0", 2L1024L1024L, 2L1024L1024L, 0}, \ {FAL_PART_MAGIC_WROD, "fdb_tsdb2", "norflash0", 4L1024L1024L, 2L1024L1024L, 0}, \ {FAL_PART_MAGIC_WROD, "fdb_tsdb3", "norflash0", 6L1024L1024L, 2L1024L1024L, 0}, \ {FAL_PART_MAGIC_WROD, "fdb_tsdb4", "norflash0", 8L1024L1024L, 2L1024L1024L, 0}, \ {FAL_PART_MAGIC_WROD, "fdb_tsdb5", "norflash0", 10L1024L1024L, 2L1024L1024L, 0}, \ {FAL_PART_MAGIC_WROD, "fdb_tsdb6", "norflash0", 12L1024L1024L, 2L1024L1024L, 0}, \ {FAL_PART_MAGIC_WROD, "fdb_tsdb7", "norflash0", 14L1024L1024L, 2L1024L1024L, 0}, \ {FAL_PART_MAGIC_WORD, "fdb_Safty", "GD32_onchip", 6001024, 401024, 0}, \ {FAL_PART_MAGIC_WORD, "fdb_Device", "GD32_onchip", 6401024, 81024, 0}, \ {FAL_PART_MAGIC_WORD, "fdb_HD00", "GD32_onchip", 6481024, 321024, 0}, \ {FAL_PART_MAGIC_WORD, "fdb_HD01", "GD32_onchip", 6801024, 321024, 0}, \ {FAL_PART_MAGIC_WORD, "fdb_ACUM", "GD32_onchip", 7121024, 161024, 0}, \ {FAL_PART_MAGIC_WORD, "fdb_Systm", "GD32_onchip", 7281024, 161024, 0}, \ {FAL_PART_MAGIC_WORD, "fdb_Event", "GD32_onchip", 7441024, 401024, 0}, \ {FAL_PART_MAGIC_WORD, "fdb_CurntE", "GD32_onchip", 7841024, 81024, 0}, \ } [10:40:05.641]收←◆[FlashDB][SYTIME] set SysTime mKey=179,0X180A--0X1210--0X001E stamp=1729267230 [FlashDB][SYTIME] Get SysTime mKey=179, 24-10-18 16:0:30

[10:40:05.822]收←◆[FlashDB][SYTIME] set SysTime mKey=180,0X180A--0X1210--0X001E stamp=1729267230 [FlashDB][SYTIME] Get SysTime mKey=180, 24-10-18 16:0:30

[10:40:06.002]收←◆[FlashDB][SYTIME] set SysTime mKey=181,0X180A--0X1210--0X001E stamp=1729267230 [FlashDB][SYTIME] Get SysTime mKey=181, 24-10-18 16:0:30

[10:40:06.182]收←◆[FlashDB][kv][SYTM][fdb_Systm] (..\module\F00302_Fdb\src\fdb_kvdb.c:929) Trigger a GC check after alloc KV failed. [FlashDB][kv][SYTM][fdb_Systm] (..\module\F00302_Fdb\src\fdb_kvdb.c:1075) Alloc an KV (size 64) failed when new KV. Now will GC then retry. [FlashDB][kv][SYTM][fdb_Systm] (..\module\F00302_Fdb\src\fdb_kvdb.c:1161) The remain empty sector is 1, GC threshold is 1. [FlashDB][kv][SYTM][fdb_Systm] (..\module\F00302_Fdb\src\fdb_kvdb.c:1130) KV (SYSTFS) is garbage NOT need move, collect it. [FlashDB][kv][SYTM][fdb_Systm] (..\module\F00302_Fdb\src\fdb_kvdb.c:1130) KV (SysTime) is garbage NOT need move, collect it.

在KV不断写入到最后一个分区时 ,不进行最后一个分区的GC ?出现以上问题后导致CPU进入硬件故障;

只保留一下两个分区:可以正常进行 GC

define FAL_PART_TABLE \

{ \ {FAL_PART_MAGIC_WORD, "fdb_Systm", "GD32_onchip", 7281024, 81024, 0}, \ {FAL_PART_MAGIC_WORD, "fdb_CurntE", "GD32_onchip", 7841024, 81024, 0}, \ }

[FlashDB][SYTIME] Get SysTime mKey=176, 24-10-18 16:0:30

[10:53:59.948]收←◆[FlashDB][SYTIME] set SysTime mKey=177,0X180A--0X1210--0X001E stamp=1729267230 [FlashDB][SYTIME] Get SysTime mKey=177, 24-10-18 16:0:30

[10:54:00.127]收←◆[FlashDB][kv][SYTM][fdb_Systm] (..\module\F00302_Fdb\src\fdb_kvdb.c:929) Trigger a GC check after alloc KV failed. [FlashDB][kv][SYTM][fdb_Systm] (..\module\F00302_Fdb\src\fdb_kvdb.c:1075) Alloc an KV (size 64) failed when new KV. Now will GC then retry. [FlashDB][kv][SYTM][fdb_Systm] (..\module\F00302_Fdb\src\fdb_kvdb.c:1161) The remain empty sector is 1, GC threshold is 1. [FlashDB][kv][SYTM][fdb_Systm] (..\module\F00302_Fdb\src\fdb_kvdb.c:1058) Moved the KV (SYSTFS) from 0x00000024 to 0x00001024. [FlashDB][kv][SYTM][fdb_Systm] (..\module\F00302_Fdb\src\fdb_kvdb.c:1130) KV (SysTime) is garbage NOT need move, collect it. [FlashDB][kv][SYTM][fdb_Systm] (..\module\F00302_Fdb\src\fdb_kvdb.c:1130) KV (SysTime) is garbage NOT need move, collect it. [FlashDB][kv][SYTM][fdb_Systm] (..\module\F00302_Fdb\src\fdb_kvdb.c:1130) KV (SysTime) is garbage NOT need move, collect it. [FlashDB][kv][SYTM][fdb_Systm] (..\module\F00302_Fdb\src\fdb_kvdb.c:1130) KV (SysTime) is garbage NOT need move, collect it. [FlashDB][kv][SYTM][fdb_Systm] (..\module\F00302_Fdb\src\fdb_kvdb.c:1130) KV (SysTime) is garbage NOT need move, collect it.

这问题很奇怪,请协助解答,非常感谢

tianzhilianwo commented 1 week ago

static const struct fal_flash_dev flash_device_find_by_part(const struct fal_partition part) { assert(part >= partition_table); assert(part <= &partition_table[partition_table_len - 1]);

return part_flash_cache[part - partition_table].flash_dev;

} 出现以上BUG是在调用该函数时出现 异常,part_flash_cache 列表的内容被修改,不知道是flashDB传过来的异常还是这个计算方式有问题?

armink commented 6 days ago

你再深入抓一下,看看哪里把内存给踩踏了?