search

zodern / meteor-up

Production Quality Meteor Deployment to Anywhere
http://meteor-up.com/
MIT License
1.27k stars 280 forks source link

Error on deploy - stat: cannot statx #1336

Closed natschoman closed 1 year ago

natschoman commented 1 year ago

I get an error ´stat: cannot statx '/built_app': Operation not permitted´ on deploying of my app. Had previously no issues, last deploy several months ago.

Mup version (mup --version): 1.5.9

Mup config

{
  "servers": {
    "one": {
      "host": "1.2.3.4",
      "username": "app_deploy",
      "password": "password"
    }
  },
  "app": {
    "name": "my-app",
    "path": "../",
    "servers": {
      "one": {}
    },
    "buildOptions": {
      "serverOnly": true,
      "buildLocation": "/var/folders/hm/yk42jlxs2c30dk6vqp26cqwm0000gn/T/mup-meteor-954b5ad7-2acd-42a2-9da5-c414c43fb608",
      "cleanBuildLocation": true
    },
    "volumes": {
      "/app_files": "/home/user1/storage",
      "/etc/timezone": "/etc/timezone",
      "/etc/localtime": "/etc/localtime"
    },
    "env": {
      "ROOT_URL": "https://subdomain.host.com",
      "MONGO_URL": "mongodb://subdomain.subdomain.host.com:27017/:27017,10.5.5.52/db?replicaSet=rs1",
      "MONGO_OPLOG_URL": "mongodb://10.5.5.51:27017,10.5.5.52:27017/local?replicaSet=rs1",
      "TZ": "Europe/Vienna",
      "VIRTUAL_HOST": "subdomain.host.com",
      "HTTPS_METHOD": "redirect",
      "LETSENCRYPT_HOST": "subdomain.host.com",
      "LETSENCRYPT_EMAIL": "email@domain.com",
      "VIRTUAL_PORT": 3000,
      "HTTP_FORWARDED_COUNT": 1
    },
    "docker": {
      "image": "zodern/meteor:latest",
      "buildInstructions": [],
      "args": [
        "-v /etc/timezone:/etc/timezone",
        "-v /etc/localtime:/etc/localtime"
      ],
      "prepareBundle": true,
      "useBuildKit": false,
      "stopAppDuringPrepareBundle": true,
      "imagePort": 3000
    },
    "enableUploadProgressBar": true,
    "type": "meteor"
  },
  "proxy": {
    "domains": "subdomain.host.com",
    "ssl": {
      "letsEncryptEmail": "email@domain.com",
      "forceSSL": true
    }
  }
}

Output of command (no errors on build steps)

...
added 153 packages from 84 contributors and audited 154 packages in 11.58s

5 packages are looking for funding

found 0 vulnerabilities

Removing intermediate container c428865f6453
 ---> ffae1ab80eb1
Successfully built ffae1ab80eb1
Successfully tagged mup-webforms.my-app.com:build

real    1m23.535s
user    0m2.056s
sys     0m1.888s
Tagged latest
Deleted Images:
deleted: sha256:581f992d64289adb2dbc2a31e03e29bec387ae5d9037de30557e843f49e23b54
deleted: sha256:1c55b8336d01a834e22e1ffe92f7d5505b9804a28ff4906e696733c67d8246ff
deleted: sha256:77baba38b8425f3b3c7111980fb200e919c9f49b3d49eda94a650c6a7fa12a1c
deleted: sha256:008dc4bdc4c7d46a5146c16fbb071cb9c8d625d07ddeb0da7f54ae3a63410ba6

Total reclaimed space: 270MB
[10.5.5.53] - Prepare Bundle: SUCCESS

Started TaskList: Configuring App
[10.5.5.53] - Pushing the Startup Script
[10.5.5.53] - Pushing the Startup Script: SUCCESS
[10.5.5.53] - Sending Environment Variables
[10.5.5.53] - Sending Environment Variables: SUCCESS

Started TaskList: Start Meteor
[10.5.5.53] - Start Meteor
using image
removing last
Image mup-webforms.my-app.com:latest
Volume
Removing docker containers. Errors about nonexistent endpoints and containers are normal.
webforms.my-app.com
Error response from daemon: endpoint webforms.my-app.com not found
Error: No such container: webforms.my-app.com-frontend
Error response from daemon: endpoint webforms.my-app.com-frontend not found
Error: No such container: webforms.my-app.com-nginx-letsencrypt
Error response from daemon: endpoint webforms.my-app.com-nginx-letsencrypt not found
Error: No such container: webforms.my-app.com-nginx-proxy
Error response from daemon: endpoint webforms.my-app.com-nginx-proxy not found
Finished removing docker containers
22136a50843fdccc65967818919475f879ec42e2a84a064a09ffc6784815757f
Ran zodern/meteor:latest
[10.5.5.53] - Start Meteor: SUCCESS
[10.5.5.53] - Verifying Deployment
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
IP: 172.17.0.2
curl: (7) Failed to connect to 172.17.0.2 port 3000: Connection refused
=> Container status:
restarted: 0 times {"Bridge":"","SandboxID":"cb6adc3e45f810ec5e7047b09e5a484e8025d20e47388627e91607aa39307bca","HairpinMode":false,"LinkLocalIPv6Address":"","LinkLocalIPv6PrefixLen":0,"Ports":{"3000/tcp":null},"SandboxKey":"/var/run/docker/netns/cb6adc3e45f8","SecondaryIPAddresses":null,"SecondaryIPv6Addresses":null,"EndpointID":"d2b4f35189684860653dd7bac144f891d9bdec1322b32776fb00684107eb0d42","Gateway":"172.17.0.1","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"IPAddress":"172.17.0.2","IPPrefixLen":16,"IPv6Gateway":"","MacAddress":"02:42:ac:11:00:02","Networks":{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"e22995ea2936c15be296fb6bb6f3c37e08a005664bcd83b4ebcd6e7986784e6c","EndpointID":"d2b4f35189684860653dd7bac144f891d9bdec1322b32776fb00684107eb0d42","Gateway":"172.17.0.1","IPAddress":"172.17.0.2","IPPrefixLen":16,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:ac:11:00:02","DriverOpts":null}}} {"Status":"running","Running":true,"Paused":false,"Restarting":false,"OOMKilled":false,"Dead":false,"Pid":7356,"ExitCode":0,"Error":"","StartedAt":"2023-01-25T16:11:58.328328195Z","FinishedAt":"0001-01-01T00:00:00Z"}
=> Logs:
stat: cannot statx '/built_app': Operation not permitted
stat: cannot statx '/built_app': Operation not permitted
HOST_UID: 
HOST_GUID: 
usermod: invalid user ID 'app'
groupmod: invalid group ID 'app'

To see more logs type 'mup logs --tail=200'

=> Redeploying previous version of the app
derwaldgeist commented 1 year ago

I am getting a similar error.

Tried to upgrade to the latest mup, but to no avail. The infrastructure setup hasn't changed. I manually deleted all containers and did a fresh mup setup and mup deploy. The result is shown below. The app is running, though. 

          ------------------------------------STDERR------------------------------------
          .4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    curl: (7) Failed to connect to 172.17.0.4 port 3000: Connection refused
    => Logs:
    => Setting node version
    NODE_VERSION=14.18.3
    v14.18.3 is already installed.
    Now using node v14.18.3 (npm v6.14.15)
    default -> 14.18.3 (-> v14.18.3 *)
    => Starting meteor app on port 3000
    Monti APM: Successfully connected
    Monti APM: completed instrumenting the app
    Google Transit: Updating Metro Rail LA
    Google Transit: Updating Metro Bus LA
    Google Transit: Processed 427 stops
    Google Transit: Processed 12023 stops

          ------------------------------------STDOUT------------------------------------
          IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    IP: 172.17.0.4
    => Container status:
    restarted: 0 times {"Bridge":"","SandboxID":"723f67e700cebdd31a423768b3db1fc1a5e95d5eeac9a672f11be50735dc967f","HairpinMode":false,"LinkLocalIPv6Address":"","LinkLocalIPv6PrefixLen":0,"Ports":{"3000/tcp":null},"SandboxKey":"/var/run/docker/netns/723f67e700ce","SecondaryIPAddresses":null,"SecondaryIPv6Addresses":null,"EndpointID":"e19f319932ad69389a3280826d72de06b4b95cfaef98c708455591ccf765a9c8","Gateway":"172.17.0.1","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"IPAddress":"172.17.0.4","IPPrefixLen":16,"IPv6Gateway":"","MacAddress":"02:42:ac:11:00:04","Networks":{"bridge":{"IPAMConfig":null,"Links":null,"Aliases":null,"NetworkID":"a9f7d05f0cbbe5f0f37029a11cda0214ec01a6393002aedeaa126a6668c3ac67","EndpointID":"e19f319932ad69389a3280826d72de06b4b95cfaef98c708455591ccf765a9c8","Gateway":"172.17.0.1","IPAddress":"172.17.0.4","IPPrefixLen":16,"IPv6Gateway":"","GlobalIPv6Address":"","GlobalIPv6PrefixLen":0,"MacAddress":"02:42:ac:11:00:04","DriverOpts":null}}} {"Status":"running","Running":true,"Paused":false,"Restarting":false,"OOMKilled":false,"Dead":false,"Pid":30706,"ExitCode":0,"Error":"","StartedAt":"2023-01-28T13:44:09.583630442Z","FinishedAt":"0001-01-01T00:00:00Z"}

    To see more logs type 'mup logs --tail=200'

          ------------------------------------------------------------------------------
derwaldgeist commented 1 year ago

Ok. I just found out that my problem was caused by the deployCheckWaitTime being set too low. Maybe you can try that as well? I raised it from 60 to 120 now.

natschoman commented 1 year ago

fyi - updated the ubuntu server and after that the deployment went through without errors.