Native NGINX support for mogstored

[GoogleCodeExporter]

Mogstored has the ability to manage starting some webservers; Apache, Lighttpd, 
or Perlbal (internal), but not Nginx.

Hoping someone can pick this up and add a patch. Currently we have instructions 
on the wiki for running nginx on a read port or write port, but it'd be pretty 
rad if mogstored could just do that for people.

Original issue reported on code.google.com by dorma...@rydia.net on 28 Feb 2012 at 2:32

[GoogleCodeExporter]

See MogileFS-Server/lib/Mogstored/HTTPServer/* for examples

Original comment by dorma...@rydia.net on 28 Feb 2012 at 2:33

[GoogleCodeExporter]

A few notes:

- I pulled some of the values for nginx from a config we use, so they may not 
be applicable, may be overkill, or underkill for a default setup (e.g. 
client_max_body_size 700M).

- nginx documentation is pretty much against running in non-daemon mode for 
production so this script will attempt to kill nginx via the pid file it 
creates and launches nginx in daemon mode.

- since nginx uses a single .tmp location (I assume this is still true) this 
script will look at the doc root and create a section in the config for each 
dev* directory found

- the attached file works well on our test environment but haven't really ran 
it though the ringer (though our configs in production are written up similar 
to this)

Added to mogstored:

< use Mogstored::HTTPServer::Nginx;

< die "Unknown server type.  Valid options: 
--server={perlbal,lighttpd,apache,nginx,none}"
<     unless $server =~ /^perlbal|lighttpd|apache|nginx|none$/;
---
> die "Unknown server type.  Valid options: 
--server={perlbal,lighttpd,apache,none}"
>     unless $server =~ /^perlbal|lighttpd|apache|none$/;

Original comment by jed.dan...@gmail.com on 28 Feb 2012 at 5:16

Attachments:

• Nginx.pm

[GoogleCodeExporter]

Note: one negative to this approach would be that adding a device would require 
a restart of mogstored on the storage server to get the new config created and 
thus making the device accessible.

Original comment by jed.dan...@gmail.com on 28 Feb 2012 at 5:32

[GoogleCodeExporter]

Cool! thanks for taking the initiative on this!

Couple comments to start:

- max_body_size should either be configurable or set to unlimited?
- If you look at a recent thread on the list (and updates to the wiki), it 
looks like disabling mkcol can speed things up quite a bit. think there's one 
other setting to that (automake dir something)
- in the DESTROY section, the `kill cat etc` is cute but you should probably 
slurp the file and use perl's kill function to send the signal :)

restarting probably isn't so bad... might also be able to send a signal to 
cycle nginx?

Original comment by dorma...@rydia.net on 28 Feb 2012 at 5:55

[GoogleCodeExporter]

Updated:

- Upped max body size to 1024M --- nginx defaults to 1M and the documentation 
doesn't make it clear what the 'unlimited' value is - or if there is one. I'll 
have to do some checking to see if 0 works

- removed mkcol, copy, and move from dav call support

- pulling in pid from the .pid file and using perl command to send signal :)

Original comment by jed.dan...@gmail.com on 28 Feb 2012 at 7:40

Attachments:

• Nginx.pm

[GoogleCodeExporter]

Attached sample config output from Nginx.pm in case someone wants to browse and 
comment on the settings without digging into the code.

Original comment by jed.dan...@gmail.com on 28 Feb 2012 at 7:45

Attachments:

• Nginx-SampleConfig.txt

[GoogleCodeExporter]

Attached version uses client_max_body_size 0; which appears to not have a limit 
in my testing. Minor other updates from there. I think this version is good for 
further field testing.

Original comment by jed.dan...@gmail.com on 1 Mar 2012 at 2:46

Attachments:

• Nginx.pm

[GoogleCodeExporter]

I've created a git branch [1] that includes Jed's latest version of Nginx.pm 
and has a couple minor fixes for how the device specific sections are generated 
and disables logging/moves the pid file to make the backend less dependent on 
the architecture of the system it is running on.

I'm starting to play around with this locally, but haven't done any production 
level testing on it yet

[1] https://github.com/frett/MogileFS-Server/tree/nginx

Original comment by daniel.frett on 3 Nov 2012 at 5:17

[GoogleCodeExporter]

Gernot Vormayr sent me a couple changes that I've incorporated into my branch 
but want to document here so others are aware of the changes.

1. tcp_nodelay is not needed since that is the default setting
2. error_page and the corresponding location section (/50x.html) are not needed
3. several nginx modules create their own temp file path (i.e. fastcgi, proxy, 
etc), to prevent these from conflicting with other running instances of nginx 
they should be localized. I accomplished this by changing the prefix directory 
to a protected .tmp dir under mogdata when starting nginx

-Daniel

Original comment by daniel.frett on 12 Nov 2012 at 7:15

[GoogleCodeExporter]

The prefix directory setting doesn't override absolute temp_file paths 
configured when nginx was built.

right now I believe the only way we can configure nginx to not trample on other 
running instances is to parse the configure arguments from "nginx -V" to 
determine which temp_paths options should be enabled.

here are the temp_path config parameters currently known about:
client_body_temp_path
fastcgi_temp_path
proxy_temp_path
scgi_temp_path
uwsgi_temp_path

-Daniel

Original comment by daniel.frett on 12 Nov 2012 at 8:26

[GoogleCodeExporter]

I have updated my branch to dynamically configure the temp_path parameters 
based on the nginx binary configure arguments.

I also have disabled daemon mode for nginx >= 1.0.9 [1] to make it easier to 
handle shutdowns/crashes

[1] http://nginx.org/en/docs/faq/daemon_master_process_off.html

Original comment by daniel.frett on 13 Nov 2012 at 7:10

[GoogleCodeExporter]

merged for next, going out in 2.67. thanks everyone!

Original comment by dorma...@rydia.net on 18 Jan 2013 at 9:37

• Changed state: Verified