search

dlundquist / sniproxy

Proxies incoming HTTP and TLS connections based on the hostname contained in the initial request of the TCP session.
BSD 2-Clause "Simplified" License
2.57k stars 398 forks source link

sniproxy keeps crashing on any 0.5 and 0.6 version #376

Open twistedr11 opened 3 years ago

twistedr11 commented 3 years ago

Hi guys,

sniproxy keeps crashing for no reason. can't find anything in logs except this ( syslog )

 message repeated 28 times: [ error on subcontainer 'ia_addr' insert (-1)]
May  6 12:07:37 uks2 systemd[1]: Created slice User Slice of root.
May  6 12:07:37 uks2 systemd[1]: Starting User Manager for UID 0...
May  6 12:07:37 uks2 systemd[1]: Started Session 11367584 of user root.
May  6 12:07:37 uks2 systemd[31867]: Reached target Sockets.
May  6 12:07:37 uks2 systemd[31867]: Reached target Paths.
May  6 12:07:37 uks2 systemd[31867]: Reached target Timers.
May  6 12:07:37 uks2 systemd[31867]: Reached target Basic System.
May  6 12:07:37 uks2 systemd[31867]: Reached target Default.
May  6 12:07:37 uks2 systemd[31867]: Startup finished in 13ms.
twistedr11 commented 3 years ago

@dlundquist

alop commented 3 years ago

Take a look at ulimit and how many open FDs SNIProxy has. I'm seeing on my systems that UDP connections are not closing. HUP'ing the process helps.

twistedr11 commented 3 years ago

HUP'ing the process

i am bit on loss here. i will post the results of ulimit on sniproxy processes, maybe u can suggest smth 

Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        0                    unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             7773                 7773                 processes 
Max open files            65536                65536                files     
Max locked memory         65536                65536                bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       7773                 7773                 signals   
Max msgqueue size         819200               819200               bytes     
Max nice priority         0                    0                    
Max realtime priority     0                    0                    
Max realtime timeout      unlimited            unlimited            us        
root@vultr:~# cat /proc/26292/limits
Limit                     Soft Limit           Hard Limit           Units     
Max cpu time              unlimited            unlimited            seconds   
Max file size             unlimited            unlimited            bytes     
Max data size             unlimited            unlimited            bytes     
Max stack size            8388608              unlimited            bytes     
Max core file size        0                    unlimited            bytes     
Max resident set          unlimited            unlimited            bytes     
Max processes             7773                 7773                 processes 
Max open files            1024                 4096                 files     
Max locked memory         65536                65536                bytes     
Max address space         unlimited            unlimited            bytes     
Max file locks            unlimited            unlimited            locks     
Max pending signals       7773                 7773                 signals   
Max msgqueue size         819200               819200               bytes     
Max nice priority         0                    0                    
Max realtime priority     0                    0                    
Max realtime timeout      unlimited            unlimited            us

it doesn't happen every time or on a certain frequency, just there are days when it keep dying every 2 hrs

alop commented 3 years ago

Yeah, you have a different issue. Mine was clearly a very low open file limit