nginx "signal process started" 进程退出问题排查

早上在上班的路同收到同事一条信息，说公司官网访问不了了。

到公司排查了下。

1

查 uptime，最近没有重启

# uptime 
 10:43:10 up 213 days, 23:38,  1 user,  load average: 0.00, 0.01, 0.05

netstat 看下，80 端口没有在监听。

~ sudo netstat -lnapt             
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      8991/sshd           
tcp        0      0 127.0.0.1:8001          0.0.0.0:*               LISTEN      22207/uwsgi         
tcp        0      0 172.18.40.217:60336     100.100.30.25:80        ESTABLISHED 26544/AliYunDun     
tcp        0      0 172.18.40.217:22        121.35.100.206:45073    ESTABLISHED 4490/sshd: sam [pri

ps aux | grep nginx 发现 nginx 进程退出了。

2 systemctl status nginx

nginx 服务 dead

# systemctl status nginx
● nginx.service - The nginx HTTP and reverse proxy server
   Loaded: loaded (/usr/lib/systemd/system/nginx.service; enabled; vendor preset: disabled)
   Active: inactive (dead) since 一 2020-04-27 01:07:41 CST; 1 day 9h ago
  Process: 31681 ExecStart=/usr/sbin/nginx (code=exited, status=0/SUCCESS)
  Process: 31677 ExecStartPre=/usr/sbin/nginx -t (code=exited, status=0/SUCCESS)
  Process: 31676 ExecStartPre=/usr/bin/rm -f /run/nginx.pid (code=exited, status=0/SUCCESS)
 Main PID: 31683 (code=exited, status=0/SUCCESS)

4月 06 01:00:32 aliyun-fmtech-me systemd[1]: Starting The nginx HTTP and reverse proxy server...
4月 06 01:00:32 aliyun-fmtech-me nginx[31677]: nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
4月 06 01:00:32 aliyun-fmtech-me nginx[31677]: nginx: configuration file /etc/nginx/nginx.conf test is successful
4月 06 01:00:32 aliyun-fmtech-me systemd[1]: Failed to parse PID from file /run/nginx.pid: Invalid argument
4月 06 01:00:32 aliyun-fmtech-me systemd[1]: Started The nginx HTTP and reverse proxy server.

3 log

到 /var/log/nginx/ 拿昨天的日志 access.log-20200427.gz 和 error.log-20200427.gz，复制到 /tmp 下 gzip -d 解压之后查看。

error.log-20200427 的最后一句是

2020/04/27 01:07:01 [notice] 2427#0: signal process started

说明 nginx 进程收到了一个 signal 信号导致退出。

查看 nginx 版本，服务器用的是 centos 默认源的

~ nginx -v
nginx version: nginx/1.16.1

查看 nginx 源码，这句打印出自 ngx_cycle.c 中的 ngx_signal_process() 函数。它的处理是到 conf 文件中找到 pid 文件路径，然后读出 pid，最后给这个 pid 发各种 signal (kill 函数)。

那么最开始这个 signal 是谁发给 nginx 进程的？为什么要发这个 signal？

meishaoming / blog

nginx "signal process started" 进程退出问题排查 #96

1

2 systemctl status nginx

3 log