ApolloAuto / apollo

An open autonomous driving platform
Apache License 2.0
24.99k stars 9.66k forks source link

共享内存扩容是不是会导致原有数据丢失? #15131

Open jiejieTop opened 12 months ago

jiejieTop commented 12 months ago
  1. 在发布者想要发布数据的时候,但此时共享内存的block大小不足以放下这个msg的时候,会出现扩容,会重新创建一个共享内存。

    bool Segment::AcquireBlockToWrite(std::size_t msg_size,
                                  WritableBlock* writable_block) {
    RETURN_VAL_IF_NULL(writable_block, false);
    if (!init_ && !OpenOrCreate()) {
    AERROR << "create shm failed, can't write now.";
    return false;
    }
    
    bool result = true;
    if (state_->need_remap()) {
    result = Remap();
    }
    
    if (msg_size > conf_.ceiling_msg_size()) {
    AINFO << "msg_size: " << msg_size
          << " larger than current shm_buffer_size: "
          << conf_.ceiling_msg_size() << " , need recreate.";
    result = Recreate(msg_size);       // 重新创建共享内存
    }
    
    if (!result) {
    AERROR << "segment update failed.";
    return false;
    }
    
    uint32_t index = GetNextWritableBlockIndex();
    writable_block->index = index;
    writable_block->block = &blocks_[index];
    writable_block->buf = block_buf_addrs_[index];
    return true;
    }
  2. 在Recreate函数中重新创建新的共享内存,扩充block的大小
    bool Segment::Recreate(const uint64_t& msg_size) {
    init_ = false;
    state_->set_need_remap(true);
    Reset();
    Remove();
    conf_.Update(msg_size);
    return OpenOrCreate();
    }
  3. 假设原有共享内存上存在订阅者还未来得及消费的数据 A 、B 、 C,此时的扩容会导致原有数据的映射丢失,旧block被重置了(没有将旧的数据库拷贝到新扩容的block上)。
  4. 订阅者想要消费数据的时候,发现共享内存需要remap,然后remap后就是新的共享内存block,无法找到之前的A 、B 、 C数据。

    bool Segment::AcquireBlockToRead(ReadableBlock* readable_block) {
    RETURN_VAL_IF_NULL(readable_block, false);
    if (!init_ && !OpenOnly()) {
    AERROR << "failed to open shared memory, can't read now.";
    return false;
    }
    
    auto index = readable_block->index;
    if (index >= conf_.block_num()) {
    AERROR << "invalid block_index[" << index << "].";
    return false;
    }
    
    bool result = true;
    if (state_->need_remap()) {
    result = Remap();                   // 需要remap,remap后没有了之前的A  B  C数据
    }
    
    if (!result) {
    AERROR << "segment update failed.";
    return false;
    }
    
    if (!blocks_[index].TryLockForRead()) {
    return false;
    }
    readable_block->block = blocks_ + index;
    readable_block->buf = block_buf_addrs_[index];
    return true;
    }
jiejieTop commented 12 months ago

相关参考:https://github.com/ApolloAuto/apollo/issues/7334 在逻辑上确实会覆盖旧的数据。