Apache with mod_neko hangs up on service reload

[maxless]

Clean Ubuntu 18.04 install with Apache 2.4.29 with mod_neko enabled. Using service reload hangs apache so I have to kill -9 it manually:

root@Ubuntu-1804-bionic-64-minimal ~ # service apache2 restart
root@Ubuntu-1804-bionic-64-minimal ~ # ps ax| grep apa
10146 ?        Ssl    0:00 /usr/sbin/apache2 -k start
10162 ?        S      0:00 /usr/sbin/apache2 -k start
10163 ?        S      0:00 /usr/sbin/apache2 -k start
10164 ?        S      0:00 /usr/sbin/apache2 -k start
10165 ?        S      0:00 /usr/sbin/apache2 -k start
10166 ?        S      0:00 /usr/sbin/apache2 -k start
10168 pts/0    S+     0:00 grep --color=auto apa
root@Ubuntu-1804-bionic-64-minimal ~ # service apache2 reload
root@Ubuntu-1804-bionic-64-minimal ~ # ps ax| grep apa
10146 ?        Ssl    0:00 /usr/sbin/apache2 -k start
10162 ?        Z      0:00 [apache2] <defunct>
10163 ?        Z      0:00 [apache2] <defunct>
10164 ?        Z      0:00 [apache2] <defunct>
10165 ?        Z      0:00 [apache2] <defunct>
10166 ?        Z      0:00 [apache2] <defunct>
10196 pts/0    S+     0:00 grep --color=auto apa
root@Ubuntu-1804-bionic-64-minimal ~ # strace -p 10146
strace: Process 10146 attached
futex(0x7f75c6a02214, FUTEX_WAIT_PRIVATE, 4294967277, NULL) = -1 EAGAIN (Resource temporarily unavailable)
futex(0x7f75c6a02214, FUTEX_WAIT_PRIVATE, 4294967267, NULL
^Cstrace: Process 10146 detached
 <detached ...>

Removing mod_neko config files allows the Apache to reload correctly.

[andyli]

Are you using mpm prefork mode? It's the only mode that Neko supports. You may reference how we config Apache for the haxelib server in https://github.com/HaxeFoundation/haxelib/blob/development/Dockerfile.

[maxless]

   Server Version: Apache/2.4.29 (Ubuntu)
   Server MPM: prefork
   Server Built: 2019-07-16T18:14:45

Server status page says I do.

From the dockerfile I think that means you use mod_tora explicitly to handle requests:

        echo 'LoadModule neko_module /usr/lib/x86_64-linux-gnu/neko/mod_neko2.ndll'; \
        echo 'LoadModule tora_module /usr/lib/x86_64-linux-gnu/neko/mod_tora2.ndll'; \
        echo 'AddHandler tora-handler .n'; \

I tested it out and, indeed, mod_tora works correctly with reload. But if I add this first line LoadModule neko..., it breaks.

[mockey]

I can confirm this. I have an old project built on mod_neko2 and I see this problem on Ubuntu 18.04 as well. systemctl reload apache2 is used e.g. by certbot when renewing letsencrypt certs, so this is not so good. Restart seems to work OK, though.

Server version: Apache/2.4.29 (Ubuntu)
Server built:   2019-09-16T12:58:48
Server MPM:     prefork
  threaded:     no
    forked:     yes (variable process count)

[maxless]

As a quick fix in my case since it was behind nginx anyways, I've just started tora server in fcgi mode (the git version iirc) and configured nginx to use it to handle index.n and removed Apache altogether, hope that helps.

[mockey]

Thanks, I'll probably do that as well. Normally I use nginx anyway nowadays. I installed apache just because of mod_neko. Does tora run reliably in fcgi mode?

[jonasmalacofilho]

I find it very reliable, yes.

Just make sure you have HaxeFoundation/tora#15 if you care about multipart/form-data (i.e. use tora from git).

[mockey]

OK, I'll give it a try. Is it possible to use PUT as well with tora?

[jonasmalacofilho]

Sorry about missing this comment @mockey.

From the top of my mind, it should. But I confess to not knowing if there's isn't anyplace that's hardcoded to ignore PUTs. But if there are, they should be easy to fix.

One thing that does come to mind though is that the only streaming API on the Haxe side is parseMultipart, which IIRC always checks for the suitable verb and headers.

[mockey]

Will try this, thanks.

[mockey]

I ran into this bug again after upgrading an old server. I suppose it won't get fixed, right? I can't switch to nginx as suggested above. It's quite annoying actually, because apache reload is used by quite a lot of programs, e.g. logrotate.