search

apache / openwhisk-apigateway

Apache OpenWhisk API Gateway service for exposing actions as REST interfaces.
https://openwhisk.apache.org/
Apache License 2.0
64 stars 45 forks source link

docker image crashes on start #385

Closed erclu closed 3 years ago

erclu commented 3 years ago

When following the instructions detailed in https://github.com/apache/openwhisk-apigateway#quick-start the container crashes with this error:

Starting api-gateway ...
/etc/init-container.sh: line 42:     8 Illegal instruction     (core dumped) /usr/local/sbin/api-gateway -V
------
2020/10/20 18:06:00 Executing sync cmd: echo ''
2020/10/20 18:06:00 Watching path /etc/api-gateway
2020/10/20 18:06:00 Watching path /etc/api-gateway/conf.d
2020/10/20 18:06:00 Watching path /etc/api-gateway/conf.d/commons
2020/10/20 18:06:00 Watching path /etc/api-gateway/conf.d/includes
2020/10/20 18:06:00 Watching path /etc/api-gateway/html
2020/10/20 18:06:00 Watching path /etc/api-gateway/scripts
2020/10/20 18:06:00 Watching path /etc/api-gateway/scripts/lua
2020/10/20 18:06:00 Watching path /etc/api-gateway/scripts/lua/lib
2020/10/20 18:06:00 Watching path /etc/api-gateway/scripts/lua/management
2020/10/20 18:06:00 Watching path /etc/api-gateway/scripts/lua/management/lib
2020/10/20 18:06:00 Watching path /etc/api-gateway/scripts/lua/management/routes
2020/10/20 18:06:00 Watching path /etc/api-gateway/scripts/lua/oauth
2020/10/20 18:06:00 Watching path /etc/api-gateway/scripts/lua/policies
2020/10/20 18:06:00 Watching path /etc/api-gateway/scripts/lua/policies/security
2020/10/20 18:06:00 Watching path /etc/api-gateway/tests
2020/10/20 18:06:00 Watching path /etc/api-gateway/tests/scripts
2020/10/20 18:06:00 Watching path /etc/api-gateway/tests/scripts/lua
2020/10/20 18:06:00 Watching path /etc/api-gateway/tests/scripts/lua/lib
2020/10/20 18:06:00 Watching path /etc/api-gateway/tests/scripts/lua/management
2020/10/20 18:06:00 Watching path /etc/api-gateway/tests/scripts/lua/management/examples
2020/10/20 18:06:00 Watching path /etc/api-gateway/tests/scripts/lua/management/lib
2020/10/20 18:06:00 Watching path /etc/api-gateway/tests/scripts/lua/policies
2020/10/20 18:06:00 Watching path /etc/api-gateway/tools
2020/10/20 18:06:00 Watching path /etc/api-gateway/tools/travis
2020/10/20 18:06:00 Starting HTTP on port 127.0.0.1:8888
sync stdout | ''
2020/10/20 18:06:00 done
2020/10/20 18:06:00 FS Changed /etc/api-gateway/conf.d/includes/resolvers.conf
2020/10/20 18:06:00 FS Changed /etc/api-gateway/conf.d/includes/resolvers.conf
   ...  with dns resolver 192.168.30.254 8.8.8.8 ipv6=off;
   ... testing configuration
/etc/init-container.sh: line 62:    24 Illegal instruction     (core dumped) api-gateway -t -p /usr/local/api-gateway/ -c /etc/api-gateway/api-gateway.conf
   ... using log level: 'warn'. Override it with -e 'LOG_LEVEL=<level>'
/etc/init-container.sh: line 73:    25 Illegal instruction     (core dumped) api-gateway -p /usr/local/api-gateway/ -c /etc/api-gateway/api-gateway.conf -g "daemon off; error_log /dev/stderr ${log_level};"
tail: can't open '/var/log/api-gateway/access.log': No such file or directory
tail: can't open '/var/log/api-gateway/gateway_error.log': No such file or directory
tail: can't open '/var/log/api-gateway/management.log': No such file or directory
2020/10/20 18:06:10 Executing sync cmd: echo ''
sync stdout | ''
2020/10/20 18:06:10 done
2020/10/20 18:06:10 Executing Reload Cmd
2020/10/20 18:06:10 Executing sync cmd: api-gateway -s reload
Error waiting for Cmd signal: illegal instruction (core dumped)
2020/10/20 18:06:10 done
sync stdout | ''

I'm talking about these lines specifically:

...
/etc/init-container.sh: line 42:     8 Illegal instruction     (core dumped) /usr/local/sbin/api-gateway -V
...
/etc/init-container.sh: line 62:    24 Illegal instruction     (core dumped) api-gateway -t -p /usr/local/api-gateway/ -c /etc/api-gateway/api-gateway.conf
...
/etc/init-container.sh: line 73:    25 Illegal instruction     (core dumped) api-gateway -p /usr/local/api-gateway/ -c /etc/api-gateway/api-gateway.conf -g "daemon off; error_log /dev/stderr ${log_level};"

this is the output when running a shell inside the container:

bash-4.4# ps aux
PID   USER     TIME  COMMAND
    1 root      0:00 /usr/bin/dumb-init -- /etc/init-container.sh
    6 root      0:00 {init-container.} /bin/bash /etc/init-container.sh
    8 root      0:00 api-gateway-config-supervisor --reload-cmd=api-gateway -s reload --sync-folder=/etc/api-gateway --sync-in
   27 root      0:00 tail -f /var/log/api-gateway/access.log -f /var/log/api-gateway/error.log -f /var/log/api-gateway/gateway
   44 root      0:00 bash
   50 root      0:00 ps aux
bash-4.4# netstat -tulp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name
tcp        0      0 localhost:8888          0.0.0.0:*               LISTEN      8/api-gateway-confi

Meaning (i guess) that the apigateway has not even started...

I'm using docker on ubuntu 20.04 server. The same problem also appears when running the openwhisk-apigateway on a microk8s kube cluster with a different container runtime, which is how I initially encountered the issue. Though strangely the whole thing works smoothly (at least regarding the apigateway) when deploying on the kubernetes cluster included with docker desktop for windows.

It could be helpful to know that the most recent tagged version that works in my docker environment is the one tagged with 0.9.0 (though using that one is not a viable solution because of another separate bug)

Might be correlated with #333 .

Version information

docker on ubuntu:

Client: Docker Engine - Community
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:02:52 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:01:20 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

kubectl on ubuntu:

Client Version: version.Info{Major:"1", Minor:"19+", GitVersion:"v1.19.2-34+1b3fa60b402c1c", GitCommit:"1b3fa60b402c1c4cb0df8a99b733ad41141a2eb7", GitTreeState:"clean", BuildDate:"2020-09-16T21:34:17Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"19+", GitVersion:"v1.19.2-34+1b3fa60b402c1c", GitCommit:"1b3fa60b402c1c4cb0df8a99b733ad41141a2eb7", GitTreeState:"clean", BuildDate:"2020-09-16T21:36:35Z", GoVersion:"go1.15.2", Compiler:"gc", Platform:"linux/amd64"}

same bug happens with microk8s v 1.18. I've tried after i saw that it worked on docker desktop for windows.

docker on windows:

Client: Docker Engine - Community
 Cloud integration  0.1.18
 Version:           19.03.13
 API version:       1.40
 Go version:        go1.13.15
 Git commit:        4484c46d9d
 Built:             Wed Sep 16 17:00:27 2020
 OS/Arch:           windows/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.13
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       4484c46d9d
  Built:            Wed Sep 16 17:07:04 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          v1.3.7
  GitCommit:        8fba4e9a7d01810a393d5d25a3621dc101981175
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683
 Kubernetes:
  Version:          v1.18.8
  StackAPI:         Unknown

kubectl on windows:

Client Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.8", GitCommit:"9f2892aab98fe339f3bd70e3c470144299398ace", GitTreeState:"clean", BuildDate:"2020-08-13T16:12:48Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"18", GitVersion:"v1.18.8", GitCommit:"9f2892aab98fe339f3bd70e3c470144299398ace", GitTreeState:"clean", BuildDate:"2020-08-13T16:04:18Z", GoVersion:"go1.13.15", Compiler:"gc", Platform:"linux/amd64"}
mhamann commented 3 years ago

@erclu is your machine a VM or running directly on bare metal? Regardless, what type/model CPU is installed?

erclu commented 3 years ago

the windows machine is a dell inspiron with an intel i7 8565U, 8 GB ram DDR4. Docker Desktop runs on WSL2 so i guess that counts as a VM.

the ubuntu machine is a physical computer with microk8s installed through snap. I think microk8s itself does not run on a VM but i'm not 100% sure on that. the processor is an older one, Intel® Pentium® Processor E5700 @ 3.00GHz with 8 GB ram DDR2

mhamann commented 3 years ago

OK, this makes sense now, and yes--likely the same problem as #333 (though we never got to the bottom of that one with the person reporting it).

Your Windows machine is a relatively modern, 8th-gen Core i7, whereas your Ubuntu machine is running on a processor that's over ten years old. I think the issue is likely that your CPU doesn't support SSE 4.2 and/or AVX, which weren't introduced until 2011. The OpenResty binaries are expecting those extensions to exist, but they don't.

You can likely re-compile and get a container that works, but I haven't tried it to know for sure. Searching around on the internet will probably help. You could try running docker build . against the API Gateway on your Ubuntu machine and perhaps you'd have success that simply, but again--not sure. :-)

Good luck!

erclu commented 3 years ago

@mhamann thank you SO MUCH for the insight, I had been dealing with this for like 2 weeks. I managed to fix pretty much everything by adding the parameter --with-luajit-xcflags='-mno-sse4.2' inside the Dockerfile. It should have some performance issues but it's not really relevant in a non-production environment which is my case.

thanks again!