search

vespa-engine / vespa

AI + Data, online. https://vespa.ai
https://vespa.ai
Apache License 2.0
5.49k stars 586 forks source link

Improve resource sampling for feed blocked #22444

Closed kkraune closed 2 years ago

kkraune commented 2 years ago

The content node will block feed when close to full, based on sampling of its environment.

This does not work in some environments, ref #22415

Suggestion from @geirst :

We should improve the memory size sampling done in proton to also include the cgroup property for memory. Currently we use:

sysconf(_SC_PHYS_PAGES) * sysconf(_SC_PAGESIZE)

We can probably take the minimum of the above and /sys/fs/cgroup/memory/memory.limit_in_bytes.

kkraune commented 2 years ago

These fixes are released in 7.586.113, thanks @toregge !

geirst commented 2 years ago

Details on total memory sampling is added to the reference doc: https://github.com/vespa-engine/documentation/pull/2054