search

influxdata / influxdb

Scalable datastore for metrics, events, and real-time analytics
https://influxdata.com
Apache License 2.0
28.71k stars 3.54k forks source link

Influxdb2 eats up 41G of memory, how to optimize it #25019

Closed lifei64 closed 4 months ago

lifei64 commented 4 months ago

Let me talk about the environment first: influxdb-version:2.0.7; go-version:1.15.2; Linux on physical machine: 64G of memory.

The problem is that now that 41G of memory is used, I'm very worried that it will take up unlimited memory and eventually cause the process to crash. The following is relevant information. It occupies 66% of the memory. The actual memory allocated to it is about 13G. It has 3 buckets and a total of 70G files are occupied. And its buff/cache usage is very high. I understand that this is the index. Regarding the memory used by data, I want to limit the high memory usage of influxdb.

image

How to deal with it? Please help me and Thanks very much!

lifei64 commented 4 months ago
image image
philjb commented 4 months ago

Check out this response on reddit.

Also, if you use a newer version you can take advantage of GOMEMLIMIT; but influxdb uses mmap so it is always going to "use" as much memory as possible for file page cache, but that is managed by the OS and high mmap shouldn't cause a out-of-memory situation. You may have an runtime: out of memory if golang cannot allocate more memory for the heap (or stack). See the top man page for more details.

lifei64 commented 4 months ago

Check out this response on reddit.

Also, if you use a newer version you can take advantage of GOMEMLIMIT; but influxdb uses mmap so it is always going to "use" as much memory as possible for file page cache, but that is managed by the OS and high mmap shouldn't cause a out-of-memory situation. You may have an runtime: out of memory if golang cannot allocate more memory for the heap (or stack). See the top man page for more details.

First of all, I am very grateful for your reply and confusion. For some reasons, I cannot open reddit well and cannot see the reply information to this question. Could you please elaborate more on how to optimize this problem, and do you have any other suggestions? Greetings to you.

philjb commented 4 months ago

I don't have anything else to suggest other than the links and information I shared above.

lifei64 commented 4 months ago

OK, thank you. I plan to upgrade the version to 2.7 and try it again.