search

xiaokai-wang / nginx-stream-upsync-module

For stream protocol. syncing upstreams from etcd or consul and so on, dynamically updating backend servers attribute, not need to reload nginx for tcp protocol, stream configure.
175 stars 48 forks source link

配置好nginx-upsync(包含nginx-stream-upsync-module，nginx-upsync-module两个模块)，error日志出现错误，求解？ #28

Closed 527593036 closed 6 years ago

527593036 commented 6 years ago

1、错误情况与现象描述如下： 其中conf/servers给了777权限，第一次启动时conf/servers目录里面是空的，启动后，生成了servers_k8s_master_apiserver.conf，servers_test.conf文件，但里面没任何内容。dump未成功。 错误日志： 2018/01/22 22:09:22 [error] 24918#0: upsync_dump_server: open dump file "/usr/local/nginx/conf/servers/servers_k8s_master_apiserver.conf" failed 2018/01/22 22:09:22 [error] 24918#0: upsync_dump_server: open dump file "/usr/local/nginx/conf/servers/servers_test.conf" failed

2、环境： 系统： centos7 64bit nginx version: nginx/1.12.2 built by gcc 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) built with OpenSSL 1.0.2k-fips 26 Jan 2017 TLS SNI support enabled configure arguments: --prefix=/usr/local/nginx-1.12.2 --with-stream --with-http_ssl_module --with-http_stub_status_module --add-module=../nginx-upsync/ --add-module=../nginx-module-vts

3、配置： http协议配置： upstream test {

fake server otherwise ngx_stream_upstream will report error when startup

server 127.0.0.1:11111;
# server 10.10.75.153:6443 max_fails=3 fail_timeout=30s weight=2;
upsync 10.10.75.151:2379/v2/keys/upstreams/k8s_master_apiserver upsync_timeout=60m upsync_interval=500ms upsync_type=etcd strong_dependency=off;
upsync_dump_path /usr/local/nginx/conf/servers/servers_test.conf;
# include /usr/local/nginx/conf/servers/servers_test.conf;

}

server { listen 8080; location = /proxy_test { proxy_pass http://test; } 

location = /upstream_show {
        upstream_show;
    }

}

tcp协议配置： upstream k8s_master_apiserver {

hash $remote_addr consistent;

# server 10.10.75.150:6443 max_fails=3 fail_timeout=30s weight=2;
# server 10.10.75.151:6443 max_fails=3 fail_timeout=30s weight=2;
# server 10.10.75.153:6443 max_fails=3 fail_timeout=30s weight=2;
server 127.0.0.1:11111;
upsync 10.10.75.150:2379/v2/keys/upstreams/k8s_master_apiserver upsync_timeout=6m upsync_interval=500ms upsync_type=etcd strong_dependency=off;
upsync_dump_path /usr/local/nginx/conf/servers/servers_k8s_master_apiserver.conf;
# include /usr/local/nginx/conf/servers/servers_k8s_master_apiserver.conf;

}

server { listen 6443; access_log /usr/local/nginx/logs/k8s.master.6443-access.log proxy; error_log /usr/local/nginx/logs/k8s.master.6443-error.log; proxy_connect_timeout 30s; proxy_timeout 30s; proxy_pass k8s_master_apiserver; proxy_next_upstream on; }

server { listen 2345; upstream_show; }

xiaokai-wang commented 6 years ago 
upstream test {
        upsync 127.0.0.1:8500/v1/kv/upstreams/test/ upsync_timeout=6m upsync_interval=500ms upsync_type=consul strong_dependency=off;
        upsync_dump_path /usr/local/nginx/conf/servers/servers_test.conf;

        include /usr/local/nginx/conf/servers/servers_test.conf;
    }

麻烦把fake server 去掉，使用include 命令，include file 里面写部分真实的server 试试；不用授权777 权限

527593036 commented 6 years ago

修改之后还是出现同样的问题 2018/01/23 13:25:51 [error] 30907#0: upsync_dump_server: open dump file "/usr/local/nginx/conf/servers/servers_test.conf" failed 2018/01/23 13:25:51 [error] 30907#0: upsync_dump_server: open dump file "/usr/local/nginx/conf/servers/servers_k8s_master_apiserver.conf" failed 2018/01/23 13:25:52 [error] 30905#0: upsync_dump_server: open dump file "/usr/local/nginx/conf/servers/servers_k8s_master_apiserver.conf" failed 2018/01/23 13:25:52 [error] 30905#0: upsync_dump_server: open dump file "/usr/local/nginx/conf/servers/servers_test.conf" failed 2018/01/23 13:25:52 [error] 30903#0: upsync_dump_server: open dump file "/usr/local/nginx/conf/servers/servers_test.conf" failed 2018/01/23 13:25:52 [error] 30903#0: upsync_dump_server: open dump file "/usr/local/nginx/conf/servers/servers_k8s_master_apiserver.conf" failed 2018/01/23 13:25:52 [error] 30904#0: upsync_dump_server: open dump file "/usr/local/nginx/conf/servers/servers_test.conf" failed 2018/01/23 13:25:52 [error] 30904#0: upsync_dump_server: open dump file "/usr/local/nginx/conf/servers/servers_k8s_master_apiserver.conf" failed

配置情况如下： [root@test75vm24 servers]# ls -alh total 12K drwxr-xr-x 2 root root 72 Jan 23 13:24 . drwx------ 5 root root 4.0K Jan 23 13:25 .. -rw-r--rw- 1 root root 64 Jan 23 13:21 servers_k8s_master_apiserver.conf -rw-r--rw- 1 root root 64 Jan 23 13:21 servers_test.conf [root@test75vm24 servers]# cat servers_test.conf server 10.10.75.153:6443 max_fails=3 fail_timeout=30s weight=2;

这个upsync_dump_path配置的逻辑是将etcd的配置落地到文件？

xiaokai-wang commented 6 years ago

upsync_dump_path 配置的逻辑是将etcd 的配置落地；

还真是没见过这个错误，在linux 系统上；配置能落地吗？

527593036 commented 6 years ago

已经解决，是权限问题，整个nginx的目录owner需要跟nginx配置文件中的user,group一致，否则就没权限写文件。修改权限后，问题解决，跑起来很正常，非常感谢，杠杠的。

xiaokai-wang commented 6 years ago

好的，谢谢