Open mikehaertl opened 6 years ago
@nikic According to #298 you seem to have a good understanding of APC's memory model. Could you maybe shed some light on the above questions?
I'm also monitoring APC with Munin and see, that the cache is frequently reset, even though the limit is not reached yet:
Any idea what could cause this?
The code to get the above numbers is basically this:
$apc = apcu_cache_info(true);
$mem = apcu_sma_info(true);
$memTotal = $mem['num_seg'] * $mem['seg_size'];
$memFree = $mem['avail_mem'];
return $this->asJson([
'memory_total' => $memTotal,
'memory_used' => $memTotal - $memFree,
'memory_vars' => $apc['mem_size'],
'cache_entries' => $apc['num_entries'],
'cache_hits' => $apc['num_hits'],
'cache_misses' => $apc['num_misses'],
]);
I would check if your webserver gets reloaded e.g. by yourself or from a cron like logrotate. A reload will reset the cache.
@schtorch I also considered that. But as you see from the graph the reset happen at random times. And actually the process runs inside a docker container and logging happens on the host system. There's no restart of php-fpm or the like.
@mikehaertl top output is notoriously hard to interpret. RES is not a good measure of per-process memory usage, because is also includes (resident) shared memory. Using RES-SHR (with a fixed cost of SHR across all processes) is not correct either, because SHR is only potentially shared memory, not all of which is necessarily shared. For APCu in particular, the full size of the 1G segment you're using will be part of VIRT (as mmapped memory), but only a part of it will be resident.
Unfortunately, none of these metrics really tell you how the actual memory usage will look like (and it's not really possible anyway). I think that (RES-SHR)N+SHR is probably the closest approximation to actual usage, e.g. in your example 216240+(230712-216240)32 = 679344, which underestimates the actual usage, but does not give the misleading results of multiplying the resident size.
In the end, I would just recommend trying out different configurations and seeing which one shows the memory usage characteristics you would like to see.
As for the apcu restarts, I'm not sure why this is happening. It does seem rather suspicious that restarts are triggered at 75% load. The reason could be that a very large memory allocation is attempted, while the memory is too fragmented to find a single continuous block of memory with sufficient size. Though I feel like this would have to be quite a huge allocation in your example. Can you please check whether you have apc.smart
set to a value other than 0? Also, do you have a cache.ttl
set?
@nikic Thanks a lot for the explanation. So there's no golden path and you always need some try & error to find the best settings. Good to know.
Regarding the restarts I don't know what's causing them. The confusing part is, that sometimes "Cache full count" increases but sometimes it reads 0 and it seems as if the process restarted (which it didn't). I have set pm.max_requests = 200
though. But I assume that APCu is managed by the master process so it should not be affected if child processes restart. But maybe I'm wrong.
Here's all the numbers from apc.php
:
General Cache Information
APCu Version | 5.1.11
PHP Version | 7.1.17
Server Software | nginx/1.12.2
Shared Memory | 1 Segment(s) with 2.0 GBytes (mmap memory)
Start Time | 2018/06/25 11:10:10
Uptime | 19 hours and 23 minutes
Cache Information
Cached Variables | 160238 ( 1.1 GBytes)
Hits | 5868414
Misses | 441800
Request Rate (hits, misses) | 90.41 cache requests/second
Hit Rate | 84.08 cache requests/second
Miss Rate | 6.33 cache requests/second
Insert Rate | 7.14 cache requests/second
Cache full count | 0
Runtime Settings
apc.coredump_unmap | 0
apc.enable_cli | 0
apc.enabled | 1
apc.entries_hint | 131072
apc.gc_ttl | 3600
apc.mmap_file_mask |
apc.preload_path |
apc.serializer | php
apc.shm_segments | 1
apc.shm_size | 2G
apc.slam_defense | 1
apc.smart | 0
apc.ttl | 0
apc.use_request_time | 1
apc.writable | /tmp
I had some hard time to figure out optimal settings for php-fpm with apc enabled. Over the years my rule of thumb was:
But with apcu it seems, this no longer is true. With a large APC memory setting RSS is way too high. Here's a real world example: I run php-fpm inside a docker container with
apc.shm_size
set to1G
. Withapc.php
I see that 400 MB are currently used (~600MB free).pm.max_children
is set to50
in php-fpm.Now if i check with
top
I get something like this:Each process seems to use ~230MB. I currently have 32 children running, so this would sum up to 32 * 230MB = 7360 MB. But if I check the real memory used by the container with
docker stats
I get this:The container only uses 809MiB. So it seems, that my calculation above was completely wrong.
Now I wonder what's a good way to get the real size of a single php-fpm process and the best setting for
pm.max_children
with APC.I think something like this should be documented somewhere.