Closed bean08 closed 1 week ago
Hi @bean08, welcome to SOFAStack community, Please sign Contributor License Agreement!
After you signed CLA, we will automatically sync the status of this pull request in 3 minutes.
The modification in question transitions the data structure for the fileMap
field from a HashMap
to a ConcurrentHashMap
within the LocalSnapshotMetaTable
class. This change enhances thread safety and concurrency control in snapshot metadata management, reflecting a focus on improving the reliability and performance of snapshot operations.
File Path | Change Summary |
---|---|
.../snapshot/local/LocalSnapshotMetaTable.java |
Swapped HashMap with ConcurrentHashMap for fileMap initialization. |
🐰✨
In the realm of code, where snapshots lay,
A change was made, one bright day.
From HashMap's embrace, we did depart,
To ConcurrentHashMap, a thread-safe art.
🌟 For concurrency, we hop and cheer,
In our code's garden, improvements appear!
🐾💻
jraft-core/src/main/java/com/alipay/sofa/jraft/storage/snapshot/local/LocalSnapshotMetaTable.java (1)
`54-54`: Switching `fileMap` to `ConcurrentHashMap` is a solid fix for the concurrency issue. Ensure to assess the performance impact and compatibility of all `fileMap` interactions.
已签署CLA但是状态一直是Required
你好,异步保存 snapshot 本身没问题,但是不需要 fileMap 支持并发,我觉得合理的做法应该是上层 join 所有 task,元数据一次写入 fileMap,否则只有部分成功的话我认为是数据损坏
你好,异步保存 snapshot 本身没问题,但是不需要 fileMap 支持并发,我觉得合理的做法应该是上层 join 所有 task,元数据一次写入 fileMap,否则只有部分成功的话我认为是数据损坏
Agree
你好,异步保存 snapshot 本身没问题,但是不需要 fileMap 支持并发,我觉得合理的做法应该是上层 join 所有 task,元数据一次写入 fileMap,否则只有部分成功的话我认为是数据损坏
感谢您的回复,我的场景是 通过jraft构建了一个框架,框架对内部各个组件的snapshot进行SPI化,SPI的实现自行决定保存的文件内容以及方式,只需要告知框架snapshot结果,框架统一对结果进行判断,如果任意SPI异常则整体异常,因此产生了并发的问题。 有理由相信其他人在未深究源码的情况下也会出现类似的问题,基于框架的鲁棒性考虑,我倒是觉得支持并发是有好处的。
Motivation:
异步保存snapshot可以提升系统性能。在异步保存snapshot时会存在 多线程并发保存不同的状态机信息到不同文件 的情况,此时com.alipay.sofa.jraft.storage.snapshot.SnapshotWriter#addFile(java.lang.String) 也是并发操作,但对于file信息的存储 com.alipay.sofa.jraft.storage.snapshot.local.LocalSnapshotMetaTable#fileMap 类型是HashMap,它是非线程安全的,在并发调用addFile会出现信息丢失的情况,目前我是靠并发调用addFile前加锁解决的。
Modification:
com.alipay.sofa.jraft.storage.snapshot.local.LocalSnapshotMetaTable#fileMap 的类型改为线程安全的 ConcurrentHashMap
Result:
Summary by CodeRabbit
ConcurrentHashMap
.