consider adding more commonly used modules to shipping nginx

[AntonioMeireles]

i have a concrete use case where i do need to consume nginx's stream module ("generic TCP/UDP proxying and load balancing"), which isn't being built/shipped by CL. current workaround is going via docker.

So, i'd like you to consider adding it. In more broader terms, i suspect that it would make sense, for CL, to build nginx - regarding the shipping modules list - in a fashion way closer to say... RHEL/CentOS 7, as it would reduce a lot of attrition/surprises when moving (or testing) workloads to CL ...

thanks in advance and best regards

[ahkok]

Agreed. Is there any way we can get a good list of say ~20-30 top nginx modules?

[ahkok]

https://www.nginx.com/resources/wiki/modules/

[ahkok]

@AntonioMeireles Where is this stream module?

[ahkok]

Ah, this is part of nginx itself, not an external module...

[AntonioMeireles]

@ahkok

currently we ship nginx built with these modules ...

bellow is the relevant part that ships with CentOS/RHEL7 ...

   --with-file-aio
    --with-ipv6 
    --with-http_ssl_module 
    --with-http_v2_module 
    --with-http_auth_request_module 
    --with-http_realip_module 
    --with-http_addition_module 
    --with-http_xslt_module=dynamic 
    --with-http_image_filter_module=dynamic 
%if 0%{?use_geoip}
    --with-http_geoip_module=dynamic 
%endif
    --with-http_sub_module 
    --with-http_dav_module 
    --with-http_flv_module 
    --with-http_mp4_module 
    --with-http_gunzip_module 
    --with-http_gzip_static_module 
    --with-http_random_index_module 
    --with-http_secure_link_module 
    --with-http_degradation_module 
    --with-http_slice_module 
    --with-http_stub_status_module 
%if 0%{?use_perl}
    --with-http_perl_module=dynamic 
%endif
    --with-mail=dynamic 
    --with-mail_ssl_module 
    --with-pcre 
    --with-pcre-jit 
    --with-stream=dynamic 
    --with-stream_ssl_module
    --with-debug 

i'd suggest enable same set (except perl and geopip which have long runtime dep tails) of modules in stock nginx pkg (and pundle) and eventually build/ship geoip and perl (as dynamic modules) in a separate non default pkg [full available set listed here]. As for the stuff that goes by default i'll leave to you side :-) to decide if to follow same pattern as RHEL regarding what modules get built inline and which ones are dynamically, or not .

All the Best,

António

[AntonioMeireles]

Hi again!

@fenrus75

many, many thanks for your initial work on this...

at sight, there's still some rough edges to polish, since as-is stream module still doesn't work out-of-the-box since as doesn't know about it (is compiled / loaded dynamically)

so, for each dynamically built module in the packaging we need to ...

• place inside a file /usr/share/nginx/conf/modules/${moduleName}.conf this snippet load_module ${fullPATHtoModule.so};

so in stream's case ... /usr/share/nginx/conf/modules/stream.conf with load_module /usr/lib64/nginx/ngx_stream_module.so; inside

and finally we need to mod the stock shipping /usr/share/nginx/conf/nginx.conf from include /etc/nginx/nginx.conf; to

include /usr/share/nginx/conf/modules/*.conf;
include /etc/nginx/nginx.conf;

that should do it.

All the best,

António

[ahkok]

@AntonioMeireles I figured as much, however, I'd like it if instead of adding them, we can just add them commented out in the template config. That way we're not enabling them by accident.

I also don't like the indirection (include an extra conf just for the load_module command). I'd just throw #load_module foo.so into the template config directly, to keep it simple.

[AntonioMeireles]

@ahkok

afaict regular policy elsewhere is to have them loaded by default (shouldn't cause any extra overload). anyway now we're entering in just plain implementation details :-)

OTOH this brings another, wider, issue - how to keep "cost of entry" into CL land as low as possible for "foreigners", due to our statelessness, and what would be the simplest/smoothest way to communicate it to the end user... i have some ideas about that but that would be for another ticket.

meanwhile thanks you all, yet again !

[AntonioMeireles]

@ahkok, btw - the indirection may make sense as a way of "future-proofing" ... think people mixing and adding to the party their own nginx modules, etc

[ahkok]

regular policy elsewhere

Umm, that's fine. But we generally think first before following the herd ;) In this case, I think there's a strong argument to be secure and minimal by default. Most extensions that require these modules will loudly error in case modules are missing, too, so I find the idea of leaving them disabled by default to be desirable myself.

may make sense as a way of "future-proofing" ... thing people mixing and adding to the party their own nginx modules

Having them commented out avoid that problem too. And people can still do this indirection themselves.

In general though, indirection is bad. It is used heavily in distros that don't use "stateless" like mechanisms to avoid pitfalls and handle renames and config upgrades and moves gracefully, but in a stateless world it makes configuration more complex and harder to maintain for the end user, so, I'm not convinced it's actually useful.