search

eclipse-iceoryx / iceoryx2

Eclipse iceoryx2™ - true zero-copy inter-process-communication in pure Rust
https://iceoryx.io
Apache License 2.0
1.03k stars 40 forks source link

[#497] resizable shared memory #517

Open elfenpiff opened 2 days ago

elfenpiff commented 2 days ago

Notes for Reviewer

This PR introduces the ResizableSharedMemory concept. It will become the basic building block for dynamic publishers and server/clients.

The basic idea is, that the ResizableSharedMemory will create a new segment for all chunks with a specific layout whenever during allocate() the current segment returns out-of-memory, alignment unsupported or size unsupported. The segment id is stored in the PointerOffset. The PointerOffset is not allowed to be larger than 64-bit because of the underlying lock-free algorithms. Therefore, the maximum supported shm size of iceoryx2 shrank to 65535 TB to make space for the 8 byte segment id.

Old segments are removed by the ResizableSharedMemory on deallocate when no more chunks are in use and the mapping is removed in the ResizableSharedMemoryView when no more chunks are registered there.

The ResizableSharedMemory will be used by the Publisher as data segment and deallocate is called whenever the Subscriber returns a sample. The ResizableSharedMemoryView is used by the Subscriber to read the incoming samples. Before reading the Subscriber has to register the sample there to map new segments and has to deregister it before returning it to the publisher to unmap old segments.

Please focus on 2 things in this review:

  1. Is the API for the ResizableSharedMemory a good fit also for future use cases?
  2. Are there some tests missing for the ResizableSharedMemory. A good way to review this, is to look at just the test names and see if there is some use case I have missed.

Pre-Review Checklist for the PR Author

  1. [x] Add sensible notes for the reviewer
  2. [x] PR title is short, expressive and meaningful
  3. [x] Relevant issues are linked in the References section
  4. [x] Every source code file has a copyright header with SPDX-License-Identifier: Apache-2.0 OR MIT
  5. [x] Branch follows the naming format (iox2-123-introduce-posix-ipc-example)
  6. [x] Commits messages are according to this guideline
    • [x] Commit messages have the issue ID ([#123] Add posix ipc example)
    • [x] Commit author matches Eclipse Contributor Agreement (and ECA is signed)
  7. [x] Tests follow the best practice for testing
  8. [x] Changelog updated in the unreleased section including API breaking changes
  9. [x] Assign PR to reviewer
  10. [x] All checks have passed (except task-list-completed)

Checklist for the PR Reviewer

Post-review Checklist for the PR Author

  1. [ ] All open points are addressed and tracked via issues

References

Closes #497

codecov[bot] commented 2 days ago

Codecov Report

Attention: Patch coverage is 88.07018% with 68 lines in your changes missing coverage. Please review.

Project coverage is 79.35%. Comparing base (2bfd0c9) to head (f574878).

Files with missing lines Patch % Lines
...ceoryx2-cal/src/resizable_shared_memory/dynamic.rs 88.67% 48 Missing :warning:
iceoryx2-cal/src/shm_allocator/mod.rs 71.42% 8 Missing :warning:
iceoryx2-bb/container/src/slotmap.rs 62.50% 6 Missing :warning:
iceoryx2-bb/container/src/semantic_string.rs 50.00% 3 Missing :warning:
iceoryx2-cal/src/shm_allocator/pool_allocator.rs 96.00% 2 Missing :warning:
iceoryx2-cal/src/shm_allocator/bump_allocator.rs 97.05% 1 Missing :warning:
Additional details and impacted files [![Impacted file tree graph](https://app.codecov.io/gh/eclipse-iceoryx/iceoryx2/pull/517/graphs/tree.svg?width=650&height=150&src=pr&token=FN3YFXTJCI&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse-iceoryx)](https://app.codecov.io/gh/eclipse-iceoryx/iceoryx2/pull/517?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse-iceoryx) ```diff @@ Coverage Diff @@ ## main #517 +/- ## ========================================== + Coverage 79.14% 79.35% +0.20% ========================================== Files 201 202 +1 Lines 24142 24699 +557 ========================================== + Hits 19108 19600 +492 - Misses 5034 5099 +65 ``` | [Files with missing lines](https://app.codecov.io/gh/eclipse-iceoryx/iceoryx2/pull/517?dropdown=coverage&src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse-iceoryx) | Coverage Δ | | |---|---|---| | [iceoryx2-cal/src/shared\_memory/common.rs](https://app.codecov.io/gh/eclipse-iceoryx/iceoryx2/pull/517?src=pr&el=tree&filepath=iceoryx2-cal%2Fsrc%2Fshared_memory%2Fcommon.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse-iceoryx#diff-aWNlb3J5eDItY2FsL3NyYy9zaGFyZWRfbWVtb3J5L2NvbW1vbi5ycw==) | `86.53% <100.00%> (+0.15%)` | :arrow_up: | | [iceoryx2-cal/src/shared\_memory/mod.rs](https://app.codecov.io/gh/eclipse-iceoryx/iceoryx2/pull/517?src=pr&el=tree&filepath=iceoryx2-cal%2Fsrc%2Fshared_memory%2Fmod.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse-iceoryx#diff-aWNlb3J5eDItY2FsL3NyYy9zaGFyZWRfbWVtb3J5L21vZC5ycw==) | `100.00% <ø> (ø)` | | | [iceoryx2-cal/src/shared\_memory\_directory/file.rs](https://app.codecov.io/gh/eclipse-iceoryx/iceoryx2/pull/517?src=pr&el=tree&filepath=iceoryx2-cal%2Fsrc%2Fshared_memory_directory%2Ffile.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse-iceoryx#diff-aWNlb3J5eDItY2FsL3NyYy9zaGFyZWRfbWVtb3J5X2RpcmVjdG9yeS9maWxlLnJz) | `86.36% <100.00%> (ø)` | | | [iceoryx2-cal/src/zero\_copy\_connection/common.rs](https://app.codecov.io/gh/eclipse-iceoryx/iceoryx2/pull/517?src=pr&el=tree&filepath=iceoryx2-cal%2Fsrc%2Fzero_copy_connection%2Fcommon.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse-iceoryx#diff-aWNlb3J5eDItY2FsL3NyYy96ZXJvX2NvcHlfY29ubmVjdGlvbi9jb21tb24ucnM=) | `91.55% <100.00%> (ø)` | | | [iceoryx2/src/port/publisher.rs](https://app.codecov.io/gh/eclipse-iceoryx/iceoryx2/pull/517?src=pr&el=tree&filepath=iceoryx2%2Fsrc%2Fport%2Fpublisher.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse-iceoryx#diff-aWNlb3J5eDIvc3JjL3BvcnQvcHVibGlzaGVyLnJz) | `84.66% <100.00%> (ø)` | | | [iceoryx2/src/port/subscriber.rs](https://app.codecov.io/gh/eclipse-iceoryx/iceoryx2/pull/517?src=pr&el=tree&filepath=iceoryx2%2Fsrc%2Fport%2Fsubscriber.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse-iceoryx#diff-aWNlb3J5eDIvc3JjL3BvcnQvc3Vic2NyaWJlci5ycw==) | `93.60% <100.00%> (ø)` | | | [iceoryx2/src/sample\_mut.rs](https://app.codecov.io/gh/eclipse-iceoryx/iceoryx2/pull/517?src=pr&el=tree&filepath=iceoryx2%2Fsrc%2Fsample_mut.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse-iceoryx#diff-aWNlb3J5eDIvc3JjL3NhbXBsZV9tdXQucnM=) | `56.25% <100.00%> (ø)` | | | [iceoryx2-cal/src/shm\_allocator/bump\_allocator.rs](https://app.codecov.io/gh/eclipse-iceoryx/iceoryx2/pull/517?src=pr&el=tree&filepath=iceoryx2-cal%2Fsrc%2Fshm_allocator%2Fbump_allocator.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse-iceoryx#diff-aWNlb3J5eDItY2FsL3NyYy9zaG1fYWxsb2NhdG9yL2J1bXBfYWxsb2NhdG9yLnJz) | `97.80% <97.05%> (-0.48%)` | :arrow_down: | | [iceoryx2-cal/src/shm\_allocator/pool\_allocator.rs](https://app.codecov.io/gh/eclipse-iceoryx/iceoryx2/pull/517?src=pr&el=tree&filepath=iceoryx2-cal%2Fsrc%2Fshm_allocator%2Fpool_allocator.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse-iceoryx#diff-aWNlb3J5eDItY2FsL3NyYy9zaG1fYWxsb2NhdG9yL3Bvb2xfYWxsb2NhdG9yLnJz) | `96.77% <96.00%> (-0.56%)` | :arrow_down: | | [iceoryx2-bb/container/src/semantic\_string.rs](https://app.codecov.io/gh/eclipse-iceoryx/iceoryx2/pull/517?src=pr&el=tree&filepath=iceoryx2-bb%2Fcontainer%2Fsrc%2Fsemantic_string.rs&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse-iceoryx#diff-aWNlb3J5eDItYmIvY29udGFpbmVyL3NyYy9zZW1hbnRpY19zdHJpbmcucnM=) | `84.42% <50.00%> (-0.73%)` | :arrow_down: | | ... and [3 more](https://app.codecov.io/gh/eclipse-iceoryx/iceoryx2/pull/517?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse-iceoryx) | | ... and [5 files with indirect coverage changes](https://app.codecov.io/gh/eclipse-iceoryx/iceoryx2/pull/517/indirect-changes?src=pr&el=tree-more&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=eclipse-iceoryx)

🚨 Try these New Features: