Container not reachable on port 9980 after startup, weird defunct lo_startmain processes in process list

ghost commented 7 years ago

I just tried running a container based on collabora/code which I plan to use with nextcloud. However, currently the container launches but I cannot reach it on port 9980 in any way (I tried directly connecting to the container's local ip on port 9980 to no avail) and when I open up a shell in the container, I have a huge list of weird defunct processes:

lool       294  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       296  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       304  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       311  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       314  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       316  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       320  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       324  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       346  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       347  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       349  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       353  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       355  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       357  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       366  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       367  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       370  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       382  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       385  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       388  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       394  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       397  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       399  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       400  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       403  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       404  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       410  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       413  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       414  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       417  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       423  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       424  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       428  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       437  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       438  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       441  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       445  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       448  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       449  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       450  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       452  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       455  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       456  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       458  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       459  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       461  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       462  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       463  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       465  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       467  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>
lool       473  0.0  0.0      0     0 ?        Z    Feb22   0:00 [lo_startmain] <defunct>

This is the docker log of the container:

Generating RSA private key, 2048 bit long modulus
........................................+++
..........+++
e is 65537 (0x10001)
Generating RSA private key, 2048 bit long modulus
..............................................................+++
.....+++
e is 65537 (0x10001)
Signature ok
subject=/C=DE/ST=BW/L=Stuttgart/O=Dummy Authority/CN=localhost
Getting CA Private Key
loolforkit version details: 2.0.3 - 2.0.3
wsd-00026-0026 17:29:10.124649 [ loolwsd ] WRN  File system of /opt/lool/child-roots/. dangerously low on disk space| wsd/LOOLWSD.cpp:305
wsd-00026-0026 17:30:10.133912 [ loolwsd ] WRN  File system of /opt/lool/child-roots/. dangerously low on disk space| wsd/LOOLWSD.cpp:305
wsd-00026-0026 17:31:10.143412 [ loolwsd ] WRN  File system of /opt/lool/child-roots/. dangerously low on disk space| wsd/LOOLWSD.cpp:305
wsd-00026-0026 17:32:10.156185 [ loolwsd ] WRN  File system of /opt/lool/child-roots/. dangerously low on disk space| wsd/LOOLWSD.cpp:305
wsd-00026-0026 17:33:10.165407 [ loolwsd ] WRN  File system of /opt/lool/child-roots/. dangerously low on disk space| wsd/LOOLWSD.cpp:305
office version details: { "ProductName": "Collabora Office", "ProductVersion": "5.1", "ProductExtension": ".10.18", "BuildId": "586626676b0aa697e153c6693a1e1407f7746989" }

RHOPKINS13 commented 7 years ago

Any updates on this? I have almost the same problem, except I don't have any warnings about low disk space in my docker logs.

# ps -A
  PID TTY          TIME CMD
    1 ?        00:00:00 sh
    7 ?        00:00:00 bash
   25 ?        00:00:00 su
   26 ?        00:00:01 loolwsd
   34 ?        00:00:19 loolforkit
   51 ?        00:00:00 loolkit
   52 ?        00:00:00 loolkit
   57 ?        00:00:00 loolkit
   62 ?        00:00:00 loolkit
   67 ?        00:00:00 loolkit
   69 ?        00:00:00 loolkit
   73 ?        00:00:00 loolkit
   78 ?        00:00:00 loolkit
   83 ?        00:00:00 loolkit
   88 ?        00:00:00 loolkit
   93 ?        00:00:00 loolkit
   98 ?        00:00:00 loolkit
  103 ?        00:00:00 loolkit
  112 ?        00:00:00 loolkit
  117 ?        00:00:00 loolkit
  122 ?        00:00:00 loolkit
  127 ?        00:00:00 loolkit
  132 ?        00:00:00 loolkit
  137 ?        00:00:00 loolkit
  142 ?        00:00:00 loolkit
  147 ?        00:00:00 loolkit
  188 ?        00:00:00 lo_startmain <defunct>
  189 ?        00:00:00 lo_startmain <defunct>
  190 ?        00:00:00 lo_startmain <defunct>
  193 ?        00:00:00 lo_startmain <defunct>
  194 ?        00:00:00 lo_startmain <defunct>
  195 ?        00:00:00 lo_startmain <defunct>
  196 ?        00:00:00 lo_startmain <defunct>
  197 ?        00:00:00 lo_startmain <defunct>
  198 ?        00:00:00 lo_startmain <defunct>
  199 ?        00:00:00 lo_startmain <defunct>
  200 ?        00:00:00 lo_startmain <defunct>
  202 ?        00:00:00 lo_startmain <defunct>
  203 ?        00:00:00 lo_startmain <defunct>
  204 ?        00:00:00 lo_startmain <defunct>
  208 ?        00:00:00 lo_startmain <defunct>
  213 ?        00:00:00 lo_startmain <defunct>
  214 ?        00:00:00 lo_startmain <defunct>
  215 ?        00:00:00 lo_startmain <defunct>
  216 ?        00:00:00 lo_startmain <defunct>
  217 ?        00:00:00 lo_startmain <defunct>
  218 ?        00:00:00 lo_startmain <defunct>
  692 ?        00:00:00 sh
  712 ?        00:00:00 ps
#

Docker Logs:

Generating RSA private key, 2048 bit long modulus
....................................+++
.........................................+++
e is 65537 (0x10001)
Generating RSA private key, 2048 bit long modulus
..........................+++
.............................+++
e is 65537 (0x10001)
Signature ok
subject=/C=DE/ST=BW/L=Stuttgart/O=Dummy Authority/CN=localhost
Getting CA Private Key
loolforkit version details: 2.0.4 - 2.0.4
office version details: { "ProductName": "Collabora Office", "ProductVersion": "5.1", "ProductExtension": ".10.21", "BuildId": "e91d2c2d59b035e40bdefac5fe06fb210180ed86" }

trigrab commented 7 years ago

I'm on the same team as @JonasT . And I tried the new version of CODE 2.1 today on the same machine he used. Now we have only a few defunct processes left:

root         1  0.0  0.0   4508   844 ?        Ss   20:21   0:00 /bin/sh -c bash start-libreoffice.sh
root         7  0.0  0.0  18392  3152 ?        S    20:21   0:00 bash start-libreoffice.sh
root        25  0.0  0.0  47332  3120 ?        S    20:21   0:00 su -c /usr/bin/loolwsd --version --o:sys_template_path=/opt/lool/systemplate --o:lo_template_path=/opt/collaboraoffice5.3 --o:child_root_path=/opt/lool/child-roots --o:file_server_root_path=/usr/share/loolw
lool        26  0.0  0.0 344084 11764 ?        Ssl  20:21   0:00 /usr/bin/loolwsd --version --o:sys_template_path=/opt/lool/systemplate --o:lo_template_path=/opt/collaboraoffice5.3 --o:child_root_path=/opt/lool/child-roots --o:file_server_root_path=/usr/share/loolwsd
lool        28  0.1  1.0 914596 165180 ?       Sl   20:21   0:00 /usr/bin/loolforkit --losubpath=lo --systemplate=/opt/lool/systemplate --lotemplate=/opt/collaboraoffice5.3 --childroot=/opt/lool/child-roots/ --clientport=9980 --masterport=9981 --version
lool        30  0.0  0.3 980456 63284 ?        Sl   20:21   0:00 /usr/bin/loolforkit --losubpath=lo --systemplate=/opt/lool/systemplate --lotemplate=/opt/collaboraoffice5.3 --childroot=/opt/lool/child-roots/ --clientport=9980 --masterport=9981 --version
lool        32  0.0  0.0      0     0 ?        Z    20:21   0:00 [lo_startmain] <defunct>
lool        36  0.0  0.3 980456 63680 ?        Sl   20:21   0:00 /usr/bin/loolforkit --losubpath=lo --systemplate=/opt/lool/systemplate --lotemplate=/opt/collaboraoffice5.3 --childroot=/opt/lool/child-roots/ --clientport=9980 --masterport=9981 --version
lool        38  0.0  0.0      0     0 ?        Z    20:21   0:00 [lo_startmain] <defunct>
root        39  0.0  0.0  18600  3504 ?        Ss   20:32   0:00 bash
root        50  0.0  0.0  34764  2984 ?        R+   20:32   0:00 ps aux

On another machine where code is working, I only have one defunct process:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   4508   680 ?        Ss   20:12   0:00 /bin/sh -c bash start-libreoffice.sh
root         7  0.0  0.0  18392  2992 ?        S    20:12   0:00 bash start-libreoffice.sh
root        25  0.0  0.0  47332  3096 ?        S    20:12   0:00 su -c /usr/bin/loolwsd --version --o:sys_template_path=/opt/lool/systemplate --o:lo_template_path=/opt/collaboraoffice5.3 --o:child_root_path=/opt/lool/child-roots --o:file_server_root_path=/usr/share/loolw
lool        26  0.0  0.0 424168 13876 ?        Ssl  20:12   0:00 /usr/bin/loolwsd --version --o:sys_template_path=/opt/lool/systemplate --o:lo_template_path=/opt/collaboraoffice5.3 --o:child_root_path=/opt/lool/child-roots --o:file_server_root_path=/usr/share/loolwsd
lool        28  0.0  1.0 914596 164988 ?       Sl   20:12   0:01 /usr/bin/loolforkit --losubpath=lo --systemplate=/opt/lool/systemplate --lotemplate=/opt/collaboraoffice5.3 --childroot=/opt/lool/child-roots/ --clientport=9980 --masterport=9981 --version
lool        44  0.0  0.3 980456 63600 ?        Sl   20:12   0:00 /usr/bin/loolforkit --losubpath=lo --systemplate=/opt/lool/systemplate --lotemplate=/opt/collaboraoffice5.3 --childroot=/opt/lool/child-roots/ --clientport=9980 --masterport=9981 --version
lool        46  0.0  0.0      0     0 ?        Z    20:12   0:00 [lo_startmain] <defunct>
root        50  0.0  0.0  18600  3428 ?        Ss   20:36   0:00 bash
root        60  0.0  0.0  34764  2996 ?        R+   20:36   0:00 ps aux

Maybe I should mention that the docker-compose file I use on the other machine looks very much the same. The only differences I could imagine have an effect on the problem are, that one is running kernel 3.16 (code is working) and one is running 4.9 (code is NOT working) Both are on debian stable.

clumsycat commented 6 years ago

same issue: cant reach by all methods. installed on Centos 7

root@917c4c805e00:/# ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.0   4500   628 ?        Ss+  Nov01   0:00 /bin/sh -c bash start-libreoffice.sh
root         7  0.0  0.0  18396  1584 ?        S+   Nov01   0:00 bash start-libreoffice.sh
root        26  0.0  0.0  47324  1616 ?        S+   Nov01   0:00 su -c /usr/bin/loolwsd --version --o:sys_template_path=/opt/lool/systemplate --o:lo_template_path=/opt/collaboraoffice5.3 --o:child_root_path=/opt/lool/child-roots --o:file_server_root_path=/usr/share/lool
lool        27  0.0  0.0 351112  7784 ?        Ssl  Nov01   0:30 /usr/bin/loolwsd --version --o:sys_template_path=/opt/lool/systemplate --o:lo_template_path=/opt/collaboraoffice5.3 --o:child_root_path=/opt/lool/child-roots --o:file_server_root_path=/usr/share/loolwsd
lool        29  0.0  0.9 1112080 306108 ?      Sl   Nov01   0:08 /usr/bin/loolforkit --losubpath=lo --systemplate=/opt/lool/systemplate --lotemplate=/opt/collaboraoffice5.3 --childroot=/opt/lool/child-roots/ --clientport=9980 --masterport=9981 --rlimits=limit_virt_mem_m
lool        31  0.0  0.6 1177808 229112 ?      Sl   Nov01   0:13 /usr/bin/loolforkit --losubpath=lo --systemplate=/opt/lool/systemplate --lotemplate=/opt/collaboraoffice5.3 --childroot=/opt/lool/child-roots/ --clientport=9980 --masterport=9981 --rlimits=limit_virt_mem_m
lool        33  0.0  0.0      0     0 ?        Z    Nov01   0:00 [lo_startmain] <defunct>
root        37  0.0  0.0  18580  2132 ?        Ss   07:29   0:00 bash
root        62  0.0  0.0  34756  1620 ?        R+   07:35   0:00 ps aux

here is my docker-compose.yml

collabora_server:
  image: collabora/code
  container_name: collabora_server
  tty:
    true
  environment:
   - domain=domain.domain.com
   - username=username
   - password=password
  cap_add:
    - MKNOD
  restart:
    always

Docker logs

Generating RSA private key, 2048 bit long modulus
..........................................................................................................................................................+++
..................................+++
e is 65537 (0x10001)
Generating RSA private key, 2048 bit long modulus
...............+++
...................+++
e is 65537 (0x10001)
Signature ok
subject=/C=DE/ST=BW/L=Stuttgart/O=Dummy Authority/CN=localhost
Getting CA Private Key
frk-00029-00029 14:43:34.458194 [ forkit ] **ERR**  Ignored setting RLIMIT_FSIZE to 18446744073709551615 bytes. (errno: Success)| common/Seccomp.cpp:284
frk-00029-00029 14:43:34.458269 [ forkit ] **ERR**  Ignored setting RLIMIT_NOFILE to 18446744073709551615 files. (errno: Success)| common/Seccomp.cpp:306
loolforkit version details: 2.1.4 - b4b777d
office version details: { "ProductName": "Collabora Office", "ProductVersion": "5.3", "ProductExtension": ".10.28", "BuildId": "d66704ff6eb7b7b2a62ea279e3599dc235c67810" }

ghost commented 6 years ago

@timar any idea on what is happening here? This makes it impossible to run collabora on the affected linux systems

FrancoisBillant commented 6 years ago

+1 I'm in the same situation with all images from 2.1.2 to 2.1.5

if i try to access to the admin console, i've got the basic auth username/password form but it's looping.

host: debian stretch docker: version 17.09.0-ce

docker run -t -d --name collabora -p 127.0.0.1:9980:9980 -e 'domain=subdomain\.domain\.tld' --network=mynetwork --cap-add MKNOD -e "username=user" -e "password=password" collabora/code:2.1.5 NB: Also tried with "-P" and "--expose 9980". I've got an nginx reverse proxy in front.

docker logs collabora

Generating RSA private key, 2048 bit long modulus
...........................................................+++
........+++
e is 65537 (0x10001)
Generating RSA private key, 2048 bit long modulus
....+++
...........................+++
e is 65537 (0x10001)
Signature ok
subject=/C=DE/ST=BW/L=Stuttgart/O=Dummy Authority/CN=localhost
Getting CA Private Key
frk-00029-00029 15:20:53.108236 [ forkit ] ERR  Ignored setting RLIMIT_FSIZE to 18446744073709551615 bytes. (errno: Success)| common/Seccomp.cpp:284
frk-00029-00029 15:20:53.108329 [ forkit ] ERR  Ignored setting RLIMIT_NOFILE to 18446744073709551615 files. (errno: Success)| common/Seccomp.cpp:306
loolforkit version details: 2.1.5 - c9a8aed
kit-00031-00029 15:20:54.909232 [ loolkit ] ERR  Failed to symlink /proc/self/smaps. Memory stats will be missing. (errno: No such file or directory)| kit/Kit.cpp:1845
office version details: { "ProductName": "Collabora Office", "ProductVersion": "5.3", "ProductExtension": ".10.30", "BuildId": "2342ab86aa31a9e7795742ba4b72ad06ad312b9e" }

docker exec collabora ps -eaf

UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 15:20 pts/0    00:00:00 /bin/sh -c bash start-libreoffice.sh
root         7     1  0 15:20 pts/0    00:00:00 bash start-libreoffice.sh
root        26     7  0 15:20 pts/0    00:00:00 su -c /usr/bin/loolwsd --version --o:sys_template_path=/opt/lool/systemplate --o:lo_template_path=/opt/collaboraoffice5.3 --o:child_root_path=/opt/lool/child-roots --o:file_server_root_path=/usr/share/loolwsd -s /bin/bash lool
lool        27    26  0 15:20 ?        00:00:00 /usr/bin/loolwsd --version --o:sys_template_path=/opt/lool/systemplate --o:lo_template_path=/opt/collaboraoffice5.3 --o:child_root_path=/opt/lool/child-roots --o:file_server_root_path=/usr/share/loolwsd
lool        29    27  0 15:20 ?        00:00:01 /usr/bin/loolforkit --losubpath=lo --systemplate=/opt/lool/systemplate --lotemplate=/opt/collaboraoffice5.3 --childroot=/opt/lool/child-roots/ --clientport=9980 --masterport=9981 --rlimits=limit_virt_mem_mb:0;limit_stack_mem_kb:8000;limit_file_size_mb:0;limit_num_open_files:0 --version
lool        31    29  0 15:20 ?        00:00:00 /usr/bin/loolforkit --losubpath=lo --systemplate=/opt/lool/systemplate --lotemplate=/opt/collaboraoffice5.3 --childroot=/opt/lool/child-roots/ --clientport=9980 --masterport=9981 --rlimits=limit_virt_mem_mb:0;limit_stack_mem_kb:8000;limit_file_size_mb:0;limit_num_open_files:0 --version
lool        33    31  0 15:20 ?        00:00:00 [lo_startmain] <defunct>
root        37     0  0 15:24 ?        00:00:00 ps -eaf

I would appreciate any information/workaround/update

jlegido commented 6 years ago

Please try this:

Start the container
Check the logs:

docker logs -f collabora_container

Until you see this message:

office version details: { "ProductName": "Collabora Office", "ProductVersion": "5.3", "ProductExtension": ".10.53", "BuildId": "cf2ac11617bce9c49c92d8153ef12b55e5b903f3" }

Get the container IP address:

docker inspect drive | grep -i ipaddr | tail -n 1 | cut -d '"' -f4

Example output:

10.47.0.18

Try below URL:

https://10.47.0.18/loleaflet/dist/admin/admin.html

svenzimmermann commented 4 years ago

Same problem here on Debian bullseye. Any solution to this?

CollaboraOnline / Docker-CODE

Container not reachable on port 9980 after startup, weird defunct lo_startmain processes in process list #3