Memory leak ?

[TheRealAstroboy]

Hi all,

for some reason Postgrest is consuming a lot of my ram on my server running:

• dockerized postgrest
• dockerized postgresql
• others docker containers

Without database activity, running top -p pgrep -d "," postgrest gives me the following result in a bash shell:

postgrest`top - 17:13:25 up 56 min, 1 user, load average: 0,17, 0,24, 0,25 Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie %Cpu(s): 3,9 us, 3,3 sy, 0,0 ni, 92,6 id, 0,0 wa, 0,0 hi, 0,2 si, 0,0 st KiB Mem : 15040056 total, 5891564 free, 4857664 used, 4290828 buff/cache KiB Swap: 0 total, 0 free, 0 used. 9765248 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 11562 root 20 0 1,001t 33960 27140 S 0,0 0,2 0:02.80 postgrest

Any hints on what I can do here? Joël

Environment

• CPU 4vcore x 2,3 GHz
• RAM 15 Go RAM
• PostgreSQL version: docker postgres:10.1
• PostgREST version: docker postgrestpostgrest (Postgrest 5.1.0 (3cef4b7))
• Operating system: Debian Stretch

[ruslantalpa]

Please provide some aditional stats like

• the same command after a restart
• try running it (for a short while) on the host (not in docker)
• how many Req/s do you have and of what type (read/write)
• are you doing a lot of writes with big payloads?

After you try/answer these things there is a better chance of figuring out your problem

On 13 Nov 2018, at 18:29, TheRealAstroboy notifications@github.com wrote:

Hi all,

for some reason Postgrest is consuming a lot of my ram on my server running:

dockerized postgrest dockerized postgresql others docker containers Without database activity, running top -p pgrep -d "," postgrest gives me the following result in a bash shell:

postgrest`top - 17:13:25 up 56 min, 1 user, load average: 0,17, 0,24, 0,25 Tasks: 1 total, 0 running, 1 sleeping, 0 stopped, 0 zombie %Cpu(s): 3,9 us, 3,3 sy, 0,0 ni, 92,6 id, 0,0 wa, 0,0 hi, 0,2 si, 0,0 st KiB Mem : 15040056 total, 5891564 free, 4857664 used, 4290828 buff/cache KiB Swap: 0 total, 0 free, 0 used. 9765248 avail Mem

PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 11562 root 20 0 1,001t 33960 27140 S 0,0 0,2 0:02.80 postgrest

Any hints on what I can do here? Joël

Environment

CPU 4vcore x 2,3 GHz RAM 15 Go RAM PostgreSQL version: docker postgres:10.1 PostgREST version: docker postgrestpostgrest (Postgrest 5.1.0 (3cef4b7)) Operating system: Debian Stretch — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub, or mute the thread.

[michelp]

All of my postgrest's regularly float around 1TB of virtual ram. I always assumed this was some effect of the haskell runtime. Note that this is just virtual, the actual resident size of the process is typically < 100MB, in your case, it looks like 34MB. I don't know the exact reason, but it's never been a problem since no physical resources are being used.

[TheRealAstroboy]

First, thanks for yours quick replies.

My memory usage stats are during idle and continues after reboot or docker restart with no transactions

19:02 - Update : Michelp, I think you're right, PostgREST memory usage stay unchanged but my server is more healthy now... I think I miss the point...

[vijfhoek]

It is indeed an effect of the Haskell runtime: https://ghc.haskell.org/trac/ghc/ticket/9706. It memory maps a load of /dev/null for its own memory management, which doesn't really have any effect on performance:

BTW, I found that I could mmap 100 TB with PROT_NONE (or even PROT_READ) and MAP_PRIVATE | MAP_ANONYMOUS | MAP_FIXED with no measurable delay, and I can start 10000 such processes at once, so there doesn't seem to be any significant cost to setting up the page table mappings (at least on my system). Not sure why that is, exactly. The VSZ column in ps looks quite funny of course :)

(source)

So yes, it looks strange, but it isn't really a problem.