search

sameersbn / docker-nextcloud

MIT License
31 stars 15 forks source link

Nginx seems unstable #15

Open sOblivionsCall opened 5 years ago

sOblivionsCall commented 5 years ago

Not much to go on here but i'm adding anything i think will help. Unfortunately the container will not stay around long enough to exec into it. I am running the kubernetes pods as defined with default values on all but the nextcloud pod. Only change is below:

        - name: NEXTCLOUD_URL
          value: 10.1.1.20

This is just a temporary setup until i get everything working as needed then i will tighten security ect. Relevant info below:

spencer@Workstation:~/git/docker-nextcloud/kubernetes$ kubectl get pod
NAME         READY   STATUS             RESTARTS   AGE
nextcloud    1/1     Running            0          11m
nginx        0/1     CrashLoopBackOff   7          11m
postgresql   1/1     Running            0          11m
spencer@Workstation:~/git/docker-nextcloud/kubernetes$ kubectl get pod
NAME         READY   STATUS    RESTARTS   AGE
nextcloud    1/1     Running   0          19m
nginx        1/1     Running   10         19m
postgresql   1/1     Running   0          19m
spencer@Workstation:~/git/docker-nextcloud/kubernetes$ kubectl logs nginx
Initializing datadir...
Installing configuration templates...
Configuring nginx...
Configuring Nextcloud virtualhost...
Starting nginx...
spencer@Workstation:~/git/docker-nextcloud/kubernetes$ kubectl logs nginx --previous
Initializing datadir...
Installing configuration templates...
Configuring nginx...
Configuring Nextcloud virtualhost...
Starting nginx...

I'm happy to get other information for you as needed but i'm not sure what else can help.

sOblivionsCall commented 5 years ago

adding more info:

$ kubectl describe pod nginx
Name:         nginx
Namespace:    default
Node:         yzen-erver/10.1.1.20
Start Time:   Thu, 04 Apr 2019 21:42:21 -0500
Labels:       name=nginx
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"name":"nginx"},"name":"nginx","namespace":"default"},"spec":{"cont...
Status:       Running
IP:           10.42.24.48
Containers:
  nginx:
    Container ID:  docker://fe6e7415c80792a70b52aafa394df5355e2e6316fc2cbf92b811233425aef68f
    Image:         sameersbn/nextcloud:13.0.4
    Image ID:      docker-pullable://sameersbn/nextcloud@sha256:6076ff7d75607439dd8234464b9526a3e63f860c0aa46677800f32f606d20e22
    Port:          80/TCP
    Host Port:     0/TCP
    Args:
      app:nginx
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Fri, 05 Apr 2019 18:16:40 -0500
      Finished:     Fri, 05 Apr 2019 18:17:10 -0500
    Ready:          False
    Restart Count:  410
    Liveness:       http-get http://:80/ delay=5s timeout=1s period=10s #success=1 #failure=3
    Environment:
      NEXTCLOUD_URL:           http://cloud.example.com
      NEXTCLOUD_PHP_FPM_HOST:  nextcloud
      NEXTCLOUD_PHP_FPM_PORT:  9000
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-g7sdp (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  default-token-g7sdp:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-g7sdp
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                     From                 Message
  ----     ------     ----                    ----                 -------
  Normal   Pulled     54m (x394 over 20h)     kubelet, yzen-erver  Container image "sameersbn/nextcloud:13.0.4" already present on machine
  Warning  Unhealthy  29m (x1207 over 20h)    kubelet, yzen-erver  Liveness probe failed: HTTP probe failed with statuscode: 400
  Warning  BackOff    4m45s (x4966 over 20h)  kubelet, yzen-erver  Back-off restarting failed container
$ docker version
Client:
 Version:           18.09.4
 API version:       1.39
 Go version:        go1.10.8
 Git commit:        d14af54266
 Built:             Wed Mar 27 18:35:44 2019
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          18.09.4
  API version:      1.39 (minimum version 1.12)
  Go version:       go1.10.8
  Git commit:       d14af54
  Built:            Wed Mar 27 18:01:48 2019
  OS/Arch:          linux/amd64
  Experimental:     false
$ docker info
Containers: 13
 Running: 1
 Paused: 0
 Stopped: 12
Images: 28
Server Version: 18.09.4
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bb71b10fd8f58240ca47fbb579b9d1028eea7c84
runc version: 2b18fe1d885ee5083ef9f0838fee39b62d653e30
init version: fec3683
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.15.0-47-generic
Operating System: Ubuntu 18.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 16
Total Memory: 15.66GiB
Name: Workstation
ID: PG5S:HKRP:IM4O:JVOL:BLPW:CL2H:Q46C:PYF4:K3ZQ:3AHC:LHAN:4EAS
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false
Product License: Community Engine

WARNING: No swap limit support
$ kubectl version
Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.0", GitCommit:"641856db18352033a0d96dbc99153fa3b27298e5", GitTreeState:"clean", BuildDate:"2019-03-25T15:53:57Z", GoVersion:"go1.12.1", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"12+", GitVersion:"v1.12.7-rancher1", GitCommit:"96573de545affa0a5db720a830b5d2bdf6e80b3b", GitTreeState:"clean", BuildDate:"2019-03-27T02:08:41Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}
sOblivionsCall commented 5 years ago

for anyone who comes across this later on, changing the liveness probe to 5 seconds for the nginx timeout seems to have resolved the issue. I must have just had a latency issue of some kind.:

      livenessProbe:
        httpGet:
          path: /
          port: 80
        initialDelaySeconds: 5
        timeoutSeconds: 5
sOblivionsCall commented 5 years ago

Scratch that, issue is not resolved:

$ kubectl describe pod nginx
Name:         nginx
Namespace:    default
Node:         yzen-erver/10.1.1.20
Start Time:   Sun, 14 Apr 2019 13:18:10 -0500
Labels:       name=nginx
Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"name":"nginx"},"name":"nginx","namespace":"default"},"spec":{"cont...
Status:       Running
IP:           10.42.198.234
Containers:
  nginx:
    Container ID:  docker://b8e31b7322189e5fb2f0834eb1fb7eee12abd06ea21e185101ac505bfa9cfe02
    Image:         sameersbn/nextcloud:13.0.4
    Image ID:      docker-pullable://sameersbn/nextcloud@sha256:6076ff7d75607439dd8234464b9526a3e63f860c0aa46677800f32f606d20e22
    Port:          80/TCP
    Host Port:     0/TCP
    Args:
      app:nginx
    State:          Waiting
      Reason:       CrashLoopBackOff
    Last State:     Terminated
      Reason:       Completed
      Exit Code:    0
      Started:      Sun, 14 Apr 2019 13:22:03 -0500
      Finished:     Sun, 14 Apr 2019 13:22:32 -0500
    Ready:          False
    Restart Count:  6
    Liveness:       http-get http://:80/ delay=5s timeout=5s period=10s #success=1 #failure=3
    Environment:
      NEXTCLOUD_URL:           http://cloud.example.com
      NEXTCLOUD_PHP_FPM_HOST:  nextcloud
      NEXTCLOUD_PHP_FPM_PORT:  9000
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-g7sdp (ro)
Conditions:
  Type              Status
  Initialized       True 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  default-token-g7sdp:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-g7sdp
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type     Reason     Age                     From                 Message
  ----     ------     ----                    ----                 -------
  Normal   Scheduled  5m53s                   default-scheduler    Successfully assigned default/nginx to yzen-erver
  Normal   Pulled     4m21s (x4 over 5m47s)   kubelet, yzen-erver  Container image "sameersbn/nextcloud:13.0.4" already present on machine
  Normal   Created    4m21s (x4 over 5m47s)   kubelet, yzen-erver  Created container
  Normal   Killing    4m21s (x3 over 5m21s)   kubelet, yzen-erver  Killing container with id docker://nginx:Container failed liveness probe.. Container will be killed and recreated.
  Normal   Started    4m20s (x4 over 5m47s)   kubelet, yzen-erver  Started container
  Warning  Unhealthy  4m11s (x10 over 5m41s)  kubelet, yzen-erver  Liveness probe failed: HTTP probe failed with statuscode: 400
  Warning  BackOff    39s (x11 over 3m21s)    kubelet, yzen-erver  Back-off restarting failed container
sOblivionsCall commented 5 years ago

It is very odd that the pod behaves perfectly find if i remove the liveness probe completely.