xiaokai-wang / nginx-stream-upsync-module

For stream protocol. syncing upstreams from etcd or consul and so on, dynamically updating backend servers attribute, not need to reload nginx for tcp protocol, stream configure.
175 stars 48 forks source link

nginx-upsync-module和nginx-stream-upsync-module模块可以同时使用么? #21

Closed Tycx2ry closed 6 years ago

Tycx2ry commented 7 years ago

nginx-stream-upsync-module 支持tcp nginx-stream-upsync-module支持http 你好,我可能项目中需要同时动态修改这两个协议,如果不能,求推荐别的方案或工具。

xiaokai-wang commented 7 years ago

可以编译整合这两个模块

wanzr commented 7 years ago

@Tycx2ry 请问 2个模块有同时编译成功么? 我在nginx-1.11.4/nginx-1.11.5 两个版本编译,单独编译一个模块都成功,同时2个模块编译都失败,编译参数 --with-stream --add-module=../nginx-upsync-module --add-module=../nginx-stream-upsync-module 。 报错内容: collect2: error: ld returned 1 exit status make[1]: [objs/nginx] Error 1 make[1]: Leaving directory `/usr/local/src/nginx-1.11.5' make: [build] Error 2 还有其他解决方案?

xiaokai-wang commented 7 years ago

https://github.com/CallMeFoxie/nginx-upsync/

@wanzr , 用这个

jianhaiqing commented 6 years ago

https://github.com/CallMeFoxie/nginx-upsync/ 这个分支代码有点旧了,能把新的合并过来么? 我也是以上两个模块合并的时候编译失败了: --add-module=/data/software/nginx/src/nginx-upsync-module-2.0.0 --add-module=/data/software/nginx/src/nginx-stream-upsync-module-1.1.0

错误提示: src/ngx_stream_upsync_module.c:304:5: error: initialization from incompatible pointer type [-Werror] ngx_stream_upsync_create_main_conf, / create main configuration / src/ngx_stream_upsync_module.c:304:5: error: (near initialization for ‘ngx_stream_upsync_module_ctx.postconfiguration’) [-Werror] src/ngx_stream_upsync_module.c:305:5: error: initialization from incompatible pointer type [-Werror] ngx_stream_upsync_init_main_conf, / init main configuration / src/ngx_stream_upsync_module.c:305:5: error: (near initialization for ‘ngx_stream_upsync_module_ctx.create_main_conf’) [-Werror] src/ngx_stream_upsync_module.c:307:5: error: initialization from incompatible pointer type [-Werror] ngx_stream_upsync_create_srv_conf, / create server configuration / src/ngx_stream_upsync_module.c:307:5: error: (near initialization for ‘ngx_stream_upsync_module_ctx.init_main_conf’) [-Werror] src/ngx_stream_upsync_module.c:309:1: error: missing initializer for field ‘merge_srv_conf’ of ‘ngx_stream_module_t’ [-Werror=missing-field-initializers] };

In file included from /data/software/nginx/src/nginx-stream-upsync-module-1.1.0/src/ngx_stream_upsync_module.c:8:0: src/stream/ngx_stream.h:237:36: note: ‘merge_srv_conf’ declared here char (merge_srv_conf)(ngx_conf_t cf, void prev, ^ /data/software/nginx/src/nginx-stream-upsync-module-1.1.0/src/ngx_stream_upsync_module.c: In function ‘ngx_stream_upsync_show’: /data/software/nginx/src/nginx-stream-upsync-module-1.1.0/src/ngx_stream_upsync_module.c:3744:9: error: implicit declaration of function ‘ngx_stream_close_connection’ [-Werror=implicit-function-declaration] ngx_stream_close_connection(s->connection);

xiaokai-wang commented 6 years ago

请使用两个模块的master 分支哈,现在并没有及时的更新版本;

你的nginx 的版本是?

jianhaiqing commented 6 years ago

nginx 1.12

jianhaiqing commented 6 years ago

nginx 1.12.1,使用了nginx-upsync-module-master 和 nginx-stream-upsync-module-master ,编译失败了 ./configure --prefix=/usr/local/nginx --sbin-path=/usr/sbin/nginx --conf-path=/usr/local/nginx/nginx.conf --error-log-path=/usr/local/nginx/log/error.log --http-log-path=/usr/local/nginx/log/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-http_ssl_module --with-http_realip_module --with-http_addition_module --with-http_sub_module --with-http_dav_module --with-http_flv_module --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_random_index_module --with-http_secure_link_module --with-http_stub_status_module --with-http_auth_request_module --with-mail --with-stream --with-mail_ssl_module --with-file-aio --with-ipv6 --with-cc-opt='-O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --with-pcre=/data/software/nginx/src/pcre-8.34 --with-zlib=/data/software/nginx/src/zlib-1.2.8 --with-openssl=/data/software/nginx/src/openssl-1.0.1g --add-module=/data/software/nginx/src/nginx-upsync-module-master --add-module=/data/software/nginx/src/ngx_cache_purge-2.3 --add-module=/data/software/nginx/src/nginx-goodies-nginx-sticky-module-ng-08a395c66e42 --add-module=/data/software/nginx/src/nginx-stream-upsync-module-master

collect2: error: ld returned 1 exit status make[1]: [objs/nginx] Error 1 make[1]: Leaving directory `/data/software/nginx/src/nginx-1.12.1' make: [build] Error 2 编译失败了;

xiaokai-wang commented 6 years ago

不能简单的同时编译这两个模块,会有冲突;需要用 https://github.com/CallMeFoxie/nginx-upsync/

我稍后试试这个, 谢谢

xiaokai-wang commented 6 years ago

@jianhaiqing

./configure --with-stream --add-module=/path/to/nginx-1.11.5/modules/nginx-upsync  --without-http_rewrite_module

你就用这条命令试试;如果出错,把错误信息贴出来

jianhaiqing commented 6 years ago

--with-stream --with-mail_ssl_module --with-file-aio --with-ipv6 --with-cc-opt='-O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --with-pcre=/data/software/nginx/src/pcre-8.34 --with-zlib=/data/software/nginx/src/zlib-1.2.8 --with-openssl=/data/software/nginx/src/openssl-1.0.1g --add-module=/data/software/nginx/src/nginx-upsync-module-6e1a9fe --add-module=/data/software/nginx/src/ngx_cache_purge-2.3 --add-module=/data/software/nginx/src/nginx-goodies-nginx-sticky-module-ng-08a395c66e42 ---with-stream --with-mail_ssl_module --with-file-aio --with-ipv6 --with-cc-opt='-O2 -g -pipe -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' --with-pcre=/data/software/nginx/src/pcre-8.34 --with-zlib=/data/software/nginx/src/zlib-1.2.8 --with-openssl=/data/software/nginx/src/openssl-1.0.1g --add-module=/data/software/nginx/src/nginx-upsync-module-6e1a9fe --add-module=/data/software/nginx/src/ngx_cache_purge-2.3 --add-module=/data/software/nginx/src/nginx-goodies-nginx-sticky-module-ng-08a395c66e42 --add-module=/data/software/nginx/src/nginx-stream-upsync-module-16cc7b4

我这里同时用了--with-stream 和 --add-module=/data/software/nginx/src/nginx-stream-upsync-module-16cc7b4 (https://github.com/CallMeFoxie/nginx-upsync/),还是会编译出错

怎么把nginx-stream-upsync 和 nginx-upsync-module 编译在一起吗?

jianhaiqing commented 6 years ago

你这里使用--with-stream 怎么能把nginx-stream-upsync 也编译进去呢?

jianhaiqing commented 6 years ago

@xiaokai-wang ./configure --with-stream --add-module=/data/software/nginx/src/nginx-upsync-module-6e1a9fe --without-http_rewrite_module nginx-1.12.1 只是编译nginx-upsync 是可以编译成功的;但我想把这两个模块同时编译进去,请问要怎么弄呢?

xiaokai-wang commented 6 years ago
./configure --with-stream --add-module=/path/to/nginx-1.11.5/modules/nginx-upsync

其中 nginx-upsync 是指的:https://github.com/CallMeFoxie/nginx-upsync/ ;包含两个模块

jianhaiqing commented 6 years ago

非常感谢,没有注意到config 文件;一直没理解对,刚同事提醒才明白;

gfrankliu commented 6 years ago

I just merged the fix in nginx-stream-upsync-module. Can you check out and try again? nginx-upsync-module和nginx-stream-upsync-module模块应该可以同时使用了。

7le commented 6 years ago

@xiaokai-wang 你好,请教一个问题,我用nginx-upsync-module和nginx-stream-upsync-module最新的master,通过nginx-upsync进行了编译。编译是成功的。但是我写完配置运行nginx的时候,提示nginx: [emerg] "upsync" directive is not allowed here in /data1/nginx/nginx/conf/nginx.conf:40

我nginx的版本是nginx version: nginx/1.9.2

配置是


#user  nobody;
worker_processes  1;

#error_log  logs/error.log;
#error_log  logs/error.log  notice;
#error_log  logs/error.log  info;

#pid        logs/nginx.pid;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    #log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
    #                  '$status $body_bytes_sent "$http_referer" '
    #                  '"$http_user_agent" "$http_x_forwarded_for"';

    #access_log  logs/access.log  main;

    sendfile        on;
    #tcp_nopush     on;

    #keepalive_timeout  0;
    keepalive_timeout  65;

    #gzip  on;
}

stream {
    upstream test {
        upsync 127.0.0.1:8500/v1/kv/upstreams/test/ upsync_timeout=6m upsync_interval=500ms upsync_type=consul strong_dependency=off;
        upsync_dump_path /usr/local/nginx/conf/servers/servers_test.conf;

        include /usr/local/nginx/conf/servers/servers_test.conf;
    }

    upstream bar {
        server 127.0.0.1:8090 weight=1 fail_timeout=10 max_fails=3;
    }

    server {
        listen 12345;

        proxy_connect_timeout 1s;
        proxy_timeout 3s;
        proxy_pass test;
    }

    server {
        listen 2345;

        upstream_show
    }

    server {
        listen 127.0.0.1:9091;

        proxy_responses 1;
        proxy_timeout 20s;
        proxy_pass bar;
    }
}

是我哪个环节出错了吗?

gfrankliu commented 6 years ago

nginx 1.9.2 不行。至少要更新到1.9.8 https://github.com/weibocom/nginx-upsync-module/issues/206

gfrankliu commented 6 years ago

Please post the output of nginx -V

On Jul 1, 2018, at 6:55 PM, silk notifications@github.com wrote:

@gfrankliu 我升级到了nginx version: nginx/1.9.9 但是仍然报nginx: [emerg] "upsync" directive is not allowed here in /data1/nginx/nginx/conf/nginx.conf:40

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

7le commented 6 years ago

@gfrankliu 后来成功了,非常感谢。 我先是测试了nginx version: nginx/1.9.9是有问题的,后来又试了nginx version: nginx/1.11.3也不行。最后用了nginx version: nginx/1.11.5就可以了。

strongit commented 5 years ago
./configure --with-stream --add-module=/path/to/nginx-1.11.5/modules/nginx-upsync

其中 nginx-upsync 是指的:https://github.com/CallMeFoxie/nginx-upsync/ ;包含两个模块

你好,nginx-1.11.5同时编译nginx-upsync-module和nginx-stream-upsync-module报错如下:

adding module in /app/nginx-1.11.5/nginx-upsync
: **No such file or directoryync/config: line 1: auto/have**

以下是执行的命令,cp这块需要加-r吗

tar zxvf nginx-1.11.5.tar.gz 
cp nginx-stream-upsync-module.zip nginx-upsync-module.zip nginx-upsync.zip nginx-1.11.5/
cd nginx-1.11.5
unzip nginx-stream-upsync-module.zip 
unzip nginx-upsync-module.zip 
unzip nginx-upsync.zip 
history |grep cp
cp  nginx-stream-upsync-module/* nginx-upsync/nginx-stream-upsync-module/
cp nginx-upsync-module/* nginx-upsync/nginx-upsync-module/
./configure --with-stream --add-module=/app/nginx-1.11.5/nginx-upsync
 ./configure  --without-http_rewrite_module --with-stream --add-module=/app/nginx-1.11.5/nginx-upsync
7le commented 5 years ago

@strongit 是的 需要加上 不然你解压的文件没有复制全呀

strongit commented 5 years ago

@7le 加上也是一样报错,不过已经找到解决方法了。configure没问题,make 出错

/app/nginx-1.11.5/nginx-upsync-master/nginx-stream-upsync-module/src/ngx_stream_upsync_module.c:304: error: initialization from incompatible pointer type
/app/nginx-1.11.5/nginx-upsync-master/nginx-stream-upsync-module/src/ngx_stream_upsync_module.c:305: error: initialization from incompatible pointer type
/app/nginx-1.11.5/nginx-upsync-master/nginx-stream-upsync-module/src/ngx_stream_upsync_module.c:307: error: initialization from incompatible pointer type
/app/nginx-1.11.5/nginx-upsync-master/nginx-stream-upsync-module/src/ngx_stream_upsync_module.c:309: error: missing initializer
/app/nginx-1.11.5/nginx-upsync-master/nginx-stream-upsync-module/src/ngx_stream_upsync_module.c:309: error: (near initialization for ‘ngx_stream_upsync_module_ctx.merge_srv_conf’)
/app/nginx-1.11.5/nginx-upsync-master/nginx-stream-upsync-module/src/ngx_stream_upsync_module.c: In function ‘ngx_stream_upsync_show’:
/app/nginx-1.11.5/nginx-upsync-master/nginx-stream-upsync-module/src/**ngx_stream_upsync_module.c**:3704: error: implicit declaration of function ‘ngx_stream_close_connection’
make[1]: *** [objs/addon/src/ngx_stream_upsync_module.o] Error 1
make[1]: Leaving directory `/app/nginx-1.11.5'
make: *** [build] Error 2
/configure --prefix=/usr/local/nginx-1.11.5 --without-http_rewrite_module --without-http_gzip_module  --with-stream --add-module=/app/nginx-1.11.5/nginx-upsync-master
adding module in /app/nginx-1.11.5/nginx-upsync-master
 + ngx_upsync_module was configured
creating objs/Makefile

Configuration summary
  + using PCRE library: /home/appdeploy/nginx/pcre-8.12
  + OpenSSL library is not used
  + zlib library is not used

  nginx path prefix: "/usr/local/nginx-1.11.5"
  nginx binary file: "/usr/local/nginx-1.11.5/sbin/nginx"
  nginx modules path: "/usr/local/nginx-1.11.5/modules"
  nginx configuration prefix: "/usr/local/nginx-1.11.5/conf"
  nginx configuration file: "/usr/local/nginx-1.11.5/conf/nginx.conf"
  nginx pid file: "/usr/local/nginx-1.11.5/logs/nginx.pid"
  nginx error log file: "/usr/local/nginx-1.11.5/logs/error.log"
  nginx http access log file: "/usr/local/nginx-1.11.5/logs/access.log"
  nginx http client request body temporary files: "client_body_temp"
  nginx http proxy temporary files: "proxy_temp"
  nginx http fastcgi temporary files: "fastcgi_temp"
  nginx http uwsgi temporary files: "uwsgi_temp"
  nginx http scgi temporary files: "scgi_temp"

之前configure报错是因为版本不对,需要这两个:nginx-upsync-module-6e1a9fe9117361539837efc81ac45303d7494dbe.zip nginx-stream-upsync- module-16cc7b412a99a876eeb183d98f15a10dfad8afe4.zip

gfrankliu commented 5 years ago

Try the latest release: https://github.com/weibocom/nginx-upsync-module/releases https://github.com/xiaokai-wang/nginx-stream-upsync-module/releases

strongit commented 5 years ago

@gfrankliu success

nginx version: nginx/1.11.5
built by gcc 4.4.7 20120313 (Red Hat 4.4.7-17) (GCC) 
configure arguments: --prefix=/usr/local/nginx-1.11.5 --without-http_rewrite_module --without-http_gzip_module --with-stream --add-module=/app/nginx-1.11.5/nginx-upsync-master