search

apache / skywalking

APM, Application Performance Monitoring System
https://skywalking.apache.org/
Apache License 2.0
23.77k stars 6.51k forks source link

[BanyanDB] Unified memory data structure and memory tracking #11338

Open sollhui opened 1 year ago

sollhui commented 1 year ago

Search before asking

Description

Memory Data Structure

Memory management is an important part of databases, and a unified memory structure can better achieve memory management and read/write. At the same time, memory alignment and padding can be used to optimize performance.

Memory tracking

It is necessary to monitor the allocation and recycling of memory, and reasonable strategies can be made based on the usage of memory, such as canceling certain queries to avoid OOM situations, which greatly helps with the stability of the database.

Use case

No response

Related issues

No response

Are you willing to submit a pull request to implement this on your own?

Code of Conduct

wu-sheng commented 1 year ago

@hanahmily Please be clear about the milestone.

The only known key feature for 0.5 is new file structure to replace the kv system.

sollhui commented 1 year ago

Anyone who interested in it can discuss it and have a try.

caicancai commented 10 months ago

hi. Is there a specific project design document to understand this part of the work ？

wu-sheng commented 10 months ago

hi. Is there a specific project design document to understand this part of the work ？

The project is in the dev stage, iterate fast by the initial team. The best way is getting started from reading the current project docs and codes first, and understand how it works with SkyWalking OPA/backend server.

wu-sheng commented 10 months ago

General docs are available here, https://skywalking.apache.org/docs/skywalking-banyandb/next/readme/. But it doesn't include TODO and WIP things.

caicancai commented 10 months ago

hi. Is there a specific project design document to understand this part of the work ？

The project is in the dev stage, iterate fast by the initial team. The best way is getting started from reading the current project docs and codes first, and understand how it works with SkyWalking OPA/backend server.

Thank you for your reply. I'll take a look at the code, see if I can help