search

BorisPolonsky / dify-helm

Deploy langgenious/dify, an LLM based app on kubernetes with helm chart
MIT License
177 stars 46 forks source link

[BUG] dif-sandbox goes into crashloop backoff with error: Cannot load open dependencies/python-requirements.txt when running on arm64 #77

Closed conall88 closed 3 months ago

conall88 commented 3 months ago

Describe the bug A clear and concise description of what the bug is.

To Reproduce

RUN:

helm repo add dify https://borispolonsky.github.io/dify-helm
helm repo update
helm install my-release dify/dify

I'm not passing any custom values.

targeting: K3s v1.29.3+k3s1 TuringPi RK1 (arm64)

see that sandbox pod goes into crashloopbackoff.

Expected behavior container should run normally

logs

2024/06/14 16:00:37 nodejs.go:32: [INFO]initializing nodejs runner environment...                                                                                                                                                 │
│ 2024/06/14 16:00:37 nodejs.go:91: [INFO]nodejs runner environment initialized                                                                                                                                                     │
│ 2024/06/14 16:00:37 setup.go:22: [INFO]initializing python runner environment...                                                                                                                                                  │
│ 2024/06/14 16:00:37 setup.go:35: [INFO]python runner environment initialized                                                                                                                                                      │
│ 2024/06/14 16:00:37 config.go:86: [INFO]network has been enabled                                                                                                                                                                  │
│ 2024/06/14 16:00:37 server.go:19: [INFO]config init success                                                                                                                                                                       │
│ 2024/06/14 16:00:37 server.go:23: [PANIC]failed to setup runner dependencies: open dependencies/python-requirements.txt: no such file or directory                                                                                │
│ panic:                                                                                                                                                                                                                            │
│                                                                                                                                                                                                                                   │
│ goroutine 1 [running]:                                                                                                                                                                                                            │
│ github.com/langgenius/dify-sandbox/internal/utils/log.(*Log).Panic(...)                                                                                                                                                           │
│     /home/yeuoly/dify-sandbox/internal/utils/log/core.go:55                                                                                                                                                                       │
│ github.com/langgenius/dify-sandbox/internal/utils/log.Panic({0x493ccd?, 0x16d98?}, {0x40000a7f38?, 0x14820?, 0x4000056768?})                                                                                                      │
│     /home/yeuoly/dify-sandbox/internal/utils/log/core.go:209 +0x74                                                                                                                                                                │
│ github.com/langgenius/dify-sandbox/internal/server.initConfig()                                                                                                                                                                   │
│     /home/yeuoly/dify-sandbox/internal/server/server.go:23 +0xb0                                                                                                                                                                  │
│ github.com/langgenius/dify-sandbox/internal/server.Run()                                                                                                                                                                          │
│     /home/yeuoly/dify-sandbox/internal/server/server.go:51 +0x1c                                                                                                                                                                  │
│ main.main()                                                                                                                                                                                                                       │
│     /home/yeuoly/dify-sandbox/cmd/server/main.go:6 +0x1c                                                                                                                                                                          │
│ Stream closed EOF for dify/dify-sandbox-5549d6d8f8-f5566 (sandbox)

Desktop (please complete the following information):

Additional context

 kubectl describe on the pod in question: 

Name:             dify-sandbox-5549d6d8f8-f5566                                                                                                                                                                                   │
│ Namespace:        dify                                                                                                                                                                                                            │
│ Priority:         0                                                                                                                                                                                                               │
│ Service Account:  default                                                                                                                                                                                                         │
│ Node:             turingnode1/192.168.0.50                                                                                                                                                                                        │
│ Start Time:       Fri, 14 Jun 2024 17:00:29 +0100                                                                                                                                                                                 │
│ Labels:           app.kubernetes.io/instance=dify                                                                                                                                                                                 │
│                   app.kubernetes.io/name=dify                                                                                                                                                                                     │
│                   component=sandbox                                                                                                                                                                                               │
│                   pod-template-hash=5549d6d8f8                                                                                                                                                                                    │
│ Annotations:      <none>                                                                                                                                                                                                          │
│ Status:           Running                                                                                                                                                                                                         │
│ IP:               10.42.0.173                                                                                                                                                                                                     │
│ IPs:                                                                                                                                                                                                                              │
│   IP:           10.42.0.173                                                                                                                                                                                                       │
│ Controlled By:  ReplicaSet/dify-sandbox-5549d6d8f8                                                                                                                                                                                │
│ Containers:                                                                                                                                                                                                                       │
│   sandbox:                                                                                                                                                                                                                        │
│     Container ID:   containerd://bc415e6a88d77bc63787be0904fdeae713372d999b67d27523fbeba2564d4ccf                                                                                                                                 │
│     Image:          langgenius/dify-sandbox:latest                                                                                                                                                                                │
│     Image ID:       docker.io/langgenius/dify-sandbox@sha256:09b7e87056739cbd29225779d0b0bef8942adfb33449e92ead40dc19f0f4d89e                                                                                                     │
│     Port:           8194/TCP                                                                                                                                                                                                      │
│     Host Port:      0/TCP                                                                                                                                                                                                         │
│     State:          Waiting                                                                                                                                                                                                       │
│       Reason:       CrashLoopBackOff                                                                                                                                                                                              │
│     Last State:     Terminated                                                                                                                                                                                                    │
│       Reason:       Error                                                                                                                                                                                                         │
│       Exit Code:    2                                                                                                                                                                                                             │
│       Started:      Fri, 14 Jun 2024 17:06:15 +0100                                                                                                                                                                               │
│       Finished:     Fri, 14 Jun 2024 17:06:15 +0100                                                                                                                                                                               │
│     Ready:          False                                                                                                                                                                                                         │
│     Restart Count:  6                                                                                                                                                                                                             │
│     Environment Variables from:                                                                                                                                                                                                   │
│       dify-sandbox  Secret  Optional: false                                                                                                                                                                                       │
│     Environment:                                                                                                                                                                                                                  │
│       GIN_MODE:        release                                                                                                                                                                                                    │
│       WORKER_TIMEOUT:  15                                                                                                                                                                                                         │
│     Mounts:                                                                                                                                                                                                                       │
│       /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-xxg56 (ro)
 Conditions:                                                                                                                                                                                                                       │
│   Type                        Status                                                                                                                                                                                              │
│   PodReadyToStartContainers   True                                                                                                                                                                                                │
│   Initialized                 True                                                                                                                                                                                                │
│   Ready                       False                                                                                                                                                                                               │
│   ContainersReady             False                                                                                                                                                                                               │
│   PodScheduled                True                                                                                                                                                                                                │
│ Volumes:                                                                                                                                                                                                                          │
│   kube-api-access-xxg56:                                                                                                                                                                                                          │
│     Type:                    Projected (a volume that contains injected data from multiple sources)                                                                                                                               │
│     TokenExpirationSeconds:  3607                                                                                                                                                                                                 │
│     ConfigMapName:           kube-root-ca.crt                                                                                                                                                                                     │
│     ConfigMapOptional:       <nil>                                                                                                                                                                                                │
│     DownwardAPI:             true                                                                                                                                                                                                 │
│ QoS Class:                   BestEffort                                                                                                                                                                                           │
│ Node-Selectors:              <none>                                                                                                                                                                                               │
│ Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s                                                                                                                                            │
│                              node.kubernetes.io/unreachable:NoExecute op=Exists for 300s                                                                                                                                          │
│ Events:                                                                                                                                                                                                                           │
│   Type     Reason     Age                     From               Message                                                                                                                                                          │
│   ----     ------     ----                    ----               -------                                                                                                                                                          │
│   Normal   Scheduled  9m25s                   default-scheduler  Successfully assigned dify/dify-sandbox-5549d6d8f8-f5566 to turingnode1                                                                                          │
│   Normal   Pulled     7m54s (x5 over 9m22s)   kubelet            Container image "langgenius/dify-sandbox:latest" already present on machine                                                                                      │
│   Normal   Created    7m54s (x5 over 9m22s)   kubelet            Created container sandbox                                                                                                                                        │
│   Normal   Started    7m54s (x5 over 9m22s)   kubelet            Started container sandbox                                                                                                                                        │
│   Warning  BackOff    4m17s (x24 over 9m17s)  kubelet            Back-off restarting failed container sandbox in pod dify-sandbox-5549d6d8f8-f5566_dify(86643f72-5712-4f1a-bcec-ef03a35654ab)
BorisPolonsky commented 3 months ago

By default sandbox component utilizes latest tag, you may need to check out which tag of sandbox image your actually running. Since these images are and provisioned by dify and closed sourced, this problem can't be resolved from this chart. You may need to submit a new issue in this repo .