search

ocf / puppet

Puppet config for OCF servers and lab machines
https://www.ocf.berkeley.edu/
31 stars 71 forks source link

low key oom should restart desktops #1422

Open lharriscal opened 6 months ago

lharriscal commented 6 months ago

Desktops just hang on a frozen screen if they run out of memory. Chrome likes using a lot of memory. I would argue that restarting the computer automatically is a more desirable behaviour for the end user than just freezing.

Therefore, on out of memory we now kernel panic, and on kernel panics we restart the computer after 5 seconds.

Already tested on firewhirl

etw wanted to test something else though before merging this.

kpengboy commented 6 months ago

Question: in OOM situations why doesn't the OOM killer kill the most offending process, thus solving the problem without a reboot? Can it be tuned to kill before the system freezes? What unintended side effects might there be if we kernel panic instantly on OOM?

lharriscal commented 6 months ago

Question: in OOM situations why doesn't the OOM killer kill the most offending process, thus solving the problem without a reboot? Can it be tuned to kill before the system freezes? What unintended side effects might there be if we kernel panic instantly on OOM?

You are definitely right on this so I did some investigation. I tried to replicate computer freezing on OOM, and I was able to do so. I tried on another computer and OOM killer functioned as expected. I am not sure whether it just only sometimes functions as expected or if it only works on some computers. more testing required.

However, none of these cases are hit in practice except on computers with low memory. Once we upgrade the problematic desktop, we likely wont run into this issue often.

lharriscal commented 6 months ago

Ok so I did a bit more research. Looks like it is just a few specific desktops that have this issue.

cg505 commented 6 months ago

banger PR title is it related to differences in swap?