search

upleveled / next-js-example-fall-2023-atvie

https://next-js-example-fall-2023-vienna-austria.fly.dev/
2 stars 2 forks source link

Major PostgreSQL version upgrade causes database files incompatibility #15

Closed karlhorky closed 8 months ago

karlhorky commented 9 months ago

When PostgreSQL upgrades to a new major version, the deploy fails with an database files are incompatible with server error:

  Creating folders for PostgreSQL and adding permissions for postgres user...
  PostgreSQL config file exists, starting database...
  waiting for server to start....2023-12-17 12:50:51.405 UTC [323] FATAL:  database files are incompatible with server
  2023-12-17 12:50:51.405 UTC [323] DETAIL:  The data directory was initialized by PostgreSQL version 15, which is not compatible with this version 16.1.
   stopped waiting
  pg_ctl: could not start server

Full logs of the deploy CI run

flyctl deploy --remote-only
==> Verifying app config
--> Verified app config
Validating /home/runner/work/next-js-example-fall-2023-atvie/next-js-example-fall-2023-atvie/fly.toml
Platform: machines
✓ Configuration is valid
==> Building image
Waiting for remote builder fly-builder-holy-wood-8157...
Remote builder fly-builder-holy-wood-8157 ready
==> Building image with Docker
--> docker host: 20.10.12 linux x86_64
#1 [internal] load build definition from Dockerfile
#1 transferring dockerfile: 1.45kB 0.0s done
#1 DONE 0.1s

#2 [internal] load .dockerignore
#2 transferring context: 479B 0.0s done
#2 DONE 0.0s

#3 [internal] load metadata for docker.io/library/node:lts-alpine
#3 DONE 0.6s

#4 [runner  1/13] FROM docker.io/library/node:lts-alpine@sha256:9e38d3d4117da74a643f67041c83914480b335c3bd44d37ccf5b5ad86cd715d1
#4 resolve docker.io/library/node:lts-alpine@sha256:9e38d3d4117da74a643f67041c83914480b335c3bd44d37ccf5b5ad86cd715d1 done
#4 sha256:9e38d3d4117da74a643f67041c83914480b335c3bd44d37ccf5b5ad86cd715d1 1.43kB / 1.43kB done
#4 sha256:e96618520c7db4c3e082648678ab72a49b73367b9a1e7884cf75ac30a198e454 1.16kB / 1.16kB done
#4 sha256:0dc964c4b6e5ccce8f301b0055b0b583508cfc5b5a72f7d9d642a442435f3d12 7.14kB / 7.14kB done
#4 sha256:661ff4d9561e3fd050929ee5097067c34bafc523ee60f5294a37fd08056a73ca 2.10MB / 3.41MB 0.1s
#4 sha256:89059ca18a987694854cd03bb6655f445ec9536917d1f8ac6e65d42e0f17f0dd 0B / 42.10MB 0.1s
#4 sha256:9bb776f8ddb06a17c9831ed6f4a2d08ebd68c020f3f689107ea1cbd69f327d9d 1.01MB / 2.34MB 0.1s
#4 sha256:ef0b6390b7f1188deb53ed9d02159c52676b5c019123fde638841a03a324e258 450B / 450B 0.1s done
#4 sha256:661ff4d9561e3fd050929ee5097067c34bafc523ee60f5294a37fd08056a73ca 3.41MB / 3.41MB 0.1s done
#4 sha256:89059ca18a987694854cd03bb6655f445ec9536917d1f8ac6e65d42e0f17f0dd 9.44MB / 42.10MB 0.2s
#4 sha256:9bb776f8ddb06a17c9831ed6f4a2d08ebd68c020f3f689107ea1cbd69f327d9d 2.34MB / 2.34MB 0.1s done
#4 extracting sha256:661ff4d9561e3fd050929ee5097067c34bafc523ee60f5294a37fd08056a73ca
#4 sha256:89059ca18a987694854cd03bb6655f445ec9536917d1f8ac6e65d42e0f17f0dd 24.12MB / 42.10MB 0.3s
#4 extracting sha256:661ff4d9561e3fd050929ee5097067c34bafc523ee60f5294a37fd08056a73ca 0.2s done
#4 ...

#5 [internal] load build context
#5 transferring context: 2.36MB 0.4s done
#5 DONE 0.4s

#4 [runner  1/13] FROM docker.io/library/node:lts-alpine@sha256:9e38d3d4117da74a643f67041c83914480b335c3bd44d37ccf5b5ad86cd715d1
#4 sha256:89059ca18a987694854cd03bb6655f445ec9536917d1f8ac6e65d42e0f17f0dd 29.36MB / 42.10MB 0.4s
#4 sha256:89059ca18a987694854cd03bb6655f445ec9536917d1f8ac6e65d42e0f17f0dd 42.10MB / 42.10MB 0.5s done
#4 extracting sha256:89059ca18a987694854cd03bb6655f445ec9536917d1f8ac6e65d42e0f17f0dd
#4 extracting sha256:89059ca18a987694854cd03bb6655f445ec9536917d1f8ac6e65d42e0f17f0dd 0.9s done
#4 extracting sha256:9bb776f8ddb06a17c9831ed6f4a2d08ebd68c020f3f689107ea1cbd69f327d9d
#4 extracting sha256:9bb776f8ddb06a17c9831ed6f4a2d08ebd68c020f3f689107ea1cbd69f327d9d 0.1s done
#4 extracting sha256:ef0b6390b7f1188deb53ed9d02159c52676b5c019123fde638841a03a324e258
#4 extracting sha256:ef0b6390b7f1188deb53ed9d02159c52676b5c019123fde638841a03a324e258 done
#4 DONE 1.7s

#6 [builder 2/8] RUN apk add --no-cache libc6-compat yq --repository=https://dl-cdn.alpinelinux.org/alpine/edge/community
#6 0.256 fetch https://dl-cdn.alpinelinux.org/alpine/edge/community/x86_64/APKINDEX.tar.gz
#6 2.702 fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/x86_64/APKINDEX.tar.gz
#6 2.749 fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/community/x86_64/APKINDEX.tar.gz
#6 3.304 (1/4) Installing musl-obstack (1.2.3-r2)
#6 3.307 (2/4) Installing libucontext (1.2-r2)
#6 3.313 (3/4) Installing gcompat (1.1.0-r4)
#6 3.321 (4/4) Installing yq (4.35.2-r1)
#6 3.672 Executing busybox-1.36.1-r15.trigger
#6 3.678 OK: 19 MiB in 21 packages
#6 DONE 3.8s

#7 [builder 3/8] RUN corepack enable && corepack prepare pnpm@latest --activate
#7 0.806 Preparing pnpm@latest for immediate activation...
#7 DONE 1.5s

#8 [builder 4/8] WORKDIR /app
#8 DONE 0.0s

#9 [builder 5/8] COPY . .
#9 DONE 0.0s

#10 [runner  2/13] RUN apk add bash postgresql
#0 0.295 fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/main/x86_64/APKINDEX.tar.gz
#0 2.751 fetch https://dl-cdn.alpinelinux.org/alpine/v3.19/community/x86_64/APKINDEX.tar.gz
#0 3.265 (1/18) Installing ncurses-terminfo-base (6.4_p20231125-r0)
#0 3.272 (2/18) Installing libncursesw (6.4_p20231125-r0)
#0 3.279 (3/18) Installing readline (8.2.1-r2)
#0 3.285 (4/18) Installing bash (5.2.21-r0)
#0 3.305 Executing bash-5.2.21-r0.post-install
#0 3.308 (5/18) Installing postgresql-common (1.2-r0)
#0 3.311 Executing postgresql-common-1.2-r0.pre-install
#0 3.322 (6/18) Installing lz4-libs (1.9.4-r5)
#0 3.327 (7/18) Installing libpq (16.1-r0)
#0 3.335 (8/18) Installing zstd-libs (1.5.5-r8)
#0 3.345 (9/18) Installing postgresql16-client (16.1-r0)
#0 3.386 (10/18) Installing tzdata (2023c-r1)
#0 3.491 (11/18) Installing icu-data-en (74.1-r0)
#0 3.530 Executing icu-data-en-74.1-r0.post-install
#0 3.532 *
#0 3.532 * If you need ICU with non-English locales and legacy charset support, install
#0 3.532 * package icu-data-full.
#0 3.532 *
#0 3.532 (12/18) Installing icu-libs (74.1-r0)
#0 3.574 (13/18) Installing gdbm (1.23-r1)
#0 3.580 (14/18) Installing libsasl (2.1.28-r5)
#0 3.586 (15/18) Installing libldap (2.6.6-r1)
#0 3.593 (16/18) Installing xz-libs (5.4.5-r0)
#0 3.598 (17/18) Installing libxml2 (2.11.6-r0)
#0 3.611 (18/18) Installing postgresql16 (16.1-r0)
#0 4.706 Executing postgresql16-16.1-r0.post-install
#0 4.708 *
#0 4.708 * If you want to use JIT in PostgreSQL, install postgresql16-jit or
#0 4.708 * postgresql-jit (if you didn't install specific major version of postgresql).
#0 4.708 *
#0 4.709 Executing busybox-1.36.1-r15.trigger
#0 4.713 Executing postgresql-common-1.2-r0.trigger
#0 4.767 * Setting postgresql16 as the default version
#0 5.298 OK: 42 MiB in 35 packages
#10 DONE 5.4s

#11 [runner  3/13] RUN corepack enable && corepack prepare pnpm@latest --activate
#11 ...

#12 [builder 6/8] RUN yq --inplace --output-format=json '(.dependencies = .dependencies * (.devDependencies | to_entries | map(select(.key | test("^(typescript|@types/*|eslint-config-upleveled)$"))) | from_entries)) | (.devDependencies = {})' package.json
#12 DONE 0.3s

#13 [builder 7/8] RUN pnpm install
#13 ...

#11 [runner  3/13] RUN corepack enable && corepack prepare pnpm@latest --activate
#11 0.934 Preparing pnpm@latest for immediate activation...
#11 DONE 1.5s

#14 [runner  4/13] WORKDIR /app
#14 DONE 0.0s

#13 [builder 7/8] RUN pnpm install
#13 1.699 Progress: resolved 0, reused 0, downloaded 1, added 0
#13 2.702 Progress: resolved 19, reused 0, downloaded 19, added 0
#13 3.718 Progress: resolved 20, reused 0, downloaded 20, added 0
#13 4.720 Progress: resolved 125, reused 0, downloaded 115, added 0
#13 5.735 Progress: resolved 249, reused 0, downloaded 223, added 0
#13 6.753 Progress: resolved 359, reused 0, downloaded 329, added 0
#13 7.759 Progress: resolved 634, reused 0, downloaded 605, added 0
#13 8.773 Progress: resolved 712, reused 0, downloaded 682, added 0
#13 9.704 Packages: +727
#13 9.704 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
#13 9.774 Progress: resolved 755, reused 0, downloaded 726, added 0
#13 10.19 Progress: resolved 755, reused 0, downloaded 726, added 727, done
#13 10.34 .../sharp@0.32.6/node_modules/sharp install$ (node install/libvips && node install/dll-copy && prebuild-install) || (node install/can-compile && node-gyp rebuild && node install/dll-copy)
#13 10.34 .../bcrypt@5.1.1/node_modules/bcrypt install$ node-pre-gyp install --fallback-to-build
#13 10.47 .../bcrypt@5.1.1/node_modules/bcrypt install: node-pre-gyp info it worked if it ends with ok
#13 10.47 .../bcrypt@5.1.1/node_modules/bcrypt install: node-pre-gyp info using node-pre-gyp@1.0.11
#13 10.47 .../bcrypt@5.1.1/node_modules/bcrypt install: node-pre-gyp info using node@20.10.0 | linux | x64
#13 10.50 .../sharp@0.32.6/node_modules/sharp install: sharp: Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.14.5/libvips-8.14.5-linuxmusl-x64.tar.br
#13 10.57 .../bcrypt@5.1.1/node_modules/bcrypt install: node-pre-gyp info check checked for "/app/node_modules/.pnpm/bcrypt@5.1.1/node_modules/bcrypt/lib/binding/napi-v3/bcrypt_lib.node" (not found)
#13 10.57 .../bcrypt@5.1.1/node_modules/bcrypt install: node-pre-gyp http GET https://github.com/kelektiv/node.bcrypt.js/releases/download/v5.1.1/bcrypt_lib-v5.1.1-napi-v3-linux-x64-musl.tar.gz
#13 11.10 .../bcrypt@5.1.1/node_modules/bcrypt install: node-pre-gyp info install unpacking napi-v3/bcrypt_lib.node
#13 11.13 .../bcrypt@5.1.1/node_modules/bcrypt install: node-pre-gyp info extracted file count: 1 
#13 11.13 .../bcrypt@5.1.1/node_modules/bcrypt install: node-pre-gyp info ok 
#13 11.13 .../bcrypt@5.1.1/node_modules/bcrypt install: [bcrypt] Success: "/app/node_modules/.pnpm/bcrypt@5.1.1/node_modules/bcrypt/lib/binding/napi-v3/bcrypt_lib.node" is installed via remote
#13 11.15 .../bcrypt@5.1.1/node_modules/bcrypt install: Done
#13 11.69 .../sharp@0.32.6/node_modules/sharp install: sharp: Integrity check passed for linuxmusl-x64
#13 12.49 .../sharp@0.32.6/node_modules/sharp install: Done
#13 12.56 .../esbuild@0.15.18/node_modules/esbuild postinstall$ node install.js
#13 12.66 .../esbuild@0.15.18/node_modules/esbuild postinstall: Done
#13 13.54 
#13 13.54 dependencies:
#13 13.54 + @testing-library/jest-dom 6.1.3
#13 13.54 + @testing-library/react 14.0.0
#13 13.54 + @types/bcrypt 5.0.1
#13 13.54 + bcrypt 5.1.1
#13 13.54 + dotenv 16.3.1
#13 13.54 + eslint-config-upleveled 7.3.3
#13 13.54 + jest 29.7.0
#13 13.54 + jest-environment-jsdom 29.7.0
#13 13.54 + ley 0.8.1
#13 13.54 + next 14.0.0
#13 13.54 + postgres 3.3.5
#13 13.54 + react 18.2.0
#13 13.54 + react-dom 18.2.0
#13 13.54 + sass 1.68.0
#13 13.54 + secure-json-parse 2.7.0
#13 13.54 + server-only 0.0.1
#13 13.54 + sharp 0.32.6
#13 13.54 + tsm 2.3.0
#13 13.54 + typescript 5.2.2
#13 13.54 + zod 3.22.4
#13 13.54 
#13 13.54 devDependencies: skipped because NODE_ENV is set to production
#13 13.54 
#13 13.54  WARN  Issues with peer dependencies found
#13 13.54 .
#13 13.54 └─┬ eslint-config-upleveled 7.3.3
#13 13.54   └── ✕ unmet peer typescript@^5.3.2: found 5.2.2
#13 13.54 
#13 13.54 Done in 13s
#13 DONE 13.8s

#15 [builder 8/8] RUN pnpm build
#15 0.861 
#15 0.861 > next-js-example-fall-2023-vienna-austria@0.1.0 build /app
#15 0.861 > next build
#15 0.861 
#15 1.640 Attention: Next.js now collects completely anonymous telemetry regarding usage.
#15 1.641 This information is used to shape Next.js' roadmap and prioritize features.
#15 1.641 You can learn more, including how to opt-out if you'd not like to participate in this anonymous program, by visiting the following URL:
#15 1.641 https://nextjs.org/telemetry
#15 1.641 
#15 1.767    ▲ Next.js 14.0.0
#15 1.767    - Environments: .env.production
#15 1.767    - Experiments (use at your own risk):
#15 1.767      · typedRoutes
#15 1.767 
#15 1.768    Creating an optimized production build ...
#15 16.23  ⚠ Compiled with warnings
#15 16.23 
#15 16.23 ./node_modules/.pnpm/next@14.0.0_@babel+core@7.23.0_react-dom@18.2.0_react@18.2.0_sass@1.68.0/node_modules/next/dist/esm/shared/lib/router/utils/app-paths.js
#15 16.23 A Node.js module is loaded ('url' at line 3) which is not supported in the Edge Runtime.
#15 16.23 Learn More: https://nextjs.org/docs/messages/node-module-in-edge-runtime
#15 16.23 
#15 16.23 Import trace for requested module:
#15 16.23 ./node_modules/.pnpm/next@14.0.0_@babel+core@7.23.0_react-dom@18.2.0_react@18.2.0_sass@1.68.0/node_modules/next/dist/esm/shared/lib/router/utils/app-paths.js
#15 16.23 
#15 16.23    Skipping validation of types
#15 16.23    Skipping linting
#15 16.60 
#15 16.60    We detected TypeScript in your project and reconfigured your tsconfig.json file for you. Strict-mode is set to false by default.
#15 16.60    The following mandatory changes were made to your tsconfig.json:
#15 16.60 
#15 16.60       - jsx was set to preserve (next.js implements its own optimized jsx transform)
#15 16.60 
#15 16.61    Collecting page data ...
#15 18.04    Generating static pages (0/21) ...
#15 18.24 
   Generating static pages (5/21) 
#15 18.40 {"name":"lukas","hobby":"coding"}
#15 18.40 { name: 'lukas', hobby: 'coding' }
#15 18.40 {"name":"lukas","hobby":"coding"}
#15 18.40 { name: 'lukas', hobby: 'coding' }
#15 18.48 
   Generating static pages (10/21) 
#15 18.61 
   Generating static pages (15/21) 
#15 18.74 
 ✓ Generating static pages (21/21) 
#15 18.91    Finalizing page optimization ...
#15 18.91    Collecting build traces ...
#15 28.86 
#15 28.88 Route (app)                                               Size     First Load JS
#15 28.88 ┌ λ /                                                     1.64 kB        94.4 kB
#15 28.88 ├ λ /_not-found                                           0 B                0 B
#15 28.88 ├ λ /about                                                157 B          87.8 kB
#15 28.88 ├ λ /animal-management-naive-dont-copy/create             157 B          87.8 kB
#15 28.88 ├ λ /animal-management-naive-dont-copy/delete/[animalId]  157 B          87.8 kB
#15 28.88 ├ λ /animal-management-naive-dont-copy/read               187 B           100 kB
#15 28.88 ├ λ /animal-management-naive-dont-copy/read/[animalId]    180 B            93 kB
#15 28.88 ├ λ /animal-management-naive-dont-copy/update/[animalId]  157 B          87.8 kB
#15 28.88 ├ λ /animals                                              187 B           100 kB
#15 28.88 ├ λ /animals-admin                                        1.34 kB          89 kB
#15 28.88 ├ λ /animals/[animalId]                                   180 B            93 kB
#15 28.88 ├ λ /animals/paginated                                    941 B          88.6 kB
#15 28.88 ├ λ /animals/with-foods/[animalId]                        180 B            93 kB
#15 28.88 ├ ○ /api                                                  0 B                0 B
#15 28.88 ├ λ /api/animals                                          0 B                0 B
#15 28.88 ├ λ /api/animals/[animalId]                               0 B                0 B
#15 28.88 ├ λ /api/login                                            0 B                0 B
#15 28.88 ├ λ /api/notes                                            0 B                0 B
#15 28.88 ├ λ /api/register                                         0 B                0 B
#15 28.88 ├ λ /cookie-example                                       1.01 kB        88.7 kB
#15 28.88 ├ λ /fruits                                               298 B            95 kB
#15 28.88 ├ λ /fruits/[fruitId]                                     1.02 kB        88.7 kB
#15 28.88 ├ λ /login                                                14.6 kB         102 kB
#15 28.88 ├ λ /notes                                                1 kB           88.7 kB
#15 28.88 ├ λ /profile/[username]                                   157 B          87.8 kB
#15 28.88 └ λ /register                                             1.11 kB        88.8 kB
#15 28.88 + First Load JS shared by all                             87.7 kB
#15 28.88   ├ chunks/328-10f28e6f60d022e7.js                        32.4 kB
#15 28.88   ├ chunks/ad84b06f-6d841a0a826ec97b.js                   53.3 kB
#15 28.88   ├ chunks/main-app-110c176c82ea0322.js                   231 B
#15 28.88   └ chunks/webpack-214f03eb53fa4809.js                    1.76 kB
#15 28.88 
#15 28.88 
#15 28.88 ƒ Middleware                                              28.4 kB
#15 28.88 
#15 28.88 ○  (Static)   prerendered as static HTML
#15 28.88 λ  (Dynamic)  server-rendered on demand using Node.js
#15 28.88 
#15 DONE 29.0s

#16 [runner  5/13] COPY --from=builder /app/.next ./.next
#16 DONE 0.1s

#17 [runner  6/13] COPY --from=builder /app/node_modules ./node_modules
#17 DONE 3.3s

#18 [runner  7/13] COPY --from=builder /app/migrations ./migrations
#18 DONE 0.0s

#19 [runner  8/13] COPY --from=builder /app/public ./public
#19 DONE 0.0s

#20 [runner  9/13] COPY --from=builder /app/package.json ./
#20 DONE 0.0s

#21 [runner 10/13] COPY --from=builder /app/.env.production ./
#21 DONE 0.0s

#22 [runner 11/13] COPY --from=builder /app/next.config.js ./
#22 DONE 0.0s

#23 [runner 12/13] COPY --from=builder /app/scripts ./scripts
#23 DONE 0.0s

#24 [runner 13/13] RUN chmod +x /app/scripts/fly-io-start.sh
#24 DONE 0.3s

#25 exporting to image
#25 exporting layers
#25 exporting layers 4.6s done
--> Building image done
==> Pushing image to fly
#25 writing image sha256:95e07bcf4bd3585cc2ce87bb435693ff50e64669b2891c713b901ff56c80c8e3 done
#25 naming to registry.fly.io/next-js-example-fall-2023-vienna-austria:deployment-01HHVX3VQRH6119MT7WEC0TT8H done
#25 DONE 4.6s
The push refers to repository [registry.fly.io/next-js-example-fall-2023-vienna-austria]
4e672c2fbcfa: Preparing
5fd3a726fbf8: Preparing
a96fc5f26865: Preparing
f910a66fcc0d: Preparing
b2babe6d4268: Preparing
fed660c96325: Preparing
5a1bff31a2c8: Preparing
9606ed592bc3: Preparing
91627420220c: Preparing
8533b1f440bf: Preparing
c01db5944501: Preparing
56607b38e451: Preparing
b256f2c132d9: Preparing
6d4bec181d96: Preparing
63247acccce3: Preparing
5af4f8f59b76: Preparing
8533b1f440bf: Waiting
91627420220c: Waiting
c01db5944501: Waiting
56607b38e451: Waiting
b256f2c132d9: Waiting
6d4bec181d96: Waiting
63247acccce3: Waiting
5af4f8f59b76: Waiting
fed660c96325: Waiting
5a1bff31a2c8: Waiting
9606ed592bc3: Waiting
4e672c2fbcfa: Pushed
b2babe6d4268: Pushed
a96fc5f26865: Pushed
f910a66fcc0d: Pushed
5fd3a726fbf8: Pushed
5a1bff31a2c8: Pushed
fed660c96325: Pushed
8533b1f440bf: Pushed
91627420220c: Pushed
c01db5944501: Pushed
b256f2c132d9: Pushed
6d4bec181d96: Pushed
5af4f8f59b76: Pushed
56607b38e451: Pushed
63247acccce3: Pushed
9606ed592bc3: Pushed
deployment-01HHVX3VQRH6119MT7WEC0TT8H: digest: sha256:f1b5428cd656f8334f86eb37018e1ff0045cf1a1a0f8fc9727e9b4d93b70373f size: 3667
--> Pushing image done
image: registry.fly.io/next-js-example-fall-2023-vienna-austria:deployment-01HHVX3VQRH6119MT7WEC0TT8H
image size: 743 MB

Watch your deployment at https://fly.io/apps/next-js-example-fall-2023-vienna-austria/monitoring

Updating existing machines in 'next-js-example-fall-2023-vienna-austria' with rolling strategy
> Updating 3d8d9299a23278 [app]
> Updating 3d8d9299a23278 [app]
> Waiting for 3d8d9299a23278 [app] to have state: started
> Machine 3d8d9299a23278 [app] has state: started
> Checking that 3d8d9299a23278 [app] is up and running
Smoke checks for 3d8d9299a23278 failed: the app appears to be crashing
Check its logs: here's the last lines below, or run 'fly logs -i 3d8d9299a23278':
  Successfully prepared image registry.fly.io/next-js-example-fall-2023-vienna-austria:deployment-01HHVX3VQRH6119MT7WEC0TT8H (21.130110969s)
  Setting up volume 'postgres'
  Opening encrypted volume
  Configuring firecracker
   INFO Sending signal SIGINT to main child process w/ PID 313
   INFO Sending signal SIGTERM to main child process w/ PID 313
   INFO Main child exited with signal (with signal 'SIGTERM', core dumped? false)
   INFO Starting clean up.
   INFO Umounting /dev/vdb from /postgres-volume
  ERROR error umounting /postgres-volume: EBUSY: Device or resource busy, retrying in a bit
  ERROR error umounting /postgres-volume: EBUSY: Device or resource busy, retrying in a bit
  ERROR error umounting /postgres-volume: EBUSY: Device or resource busy, retrying in a bit
  ERROR error umounting /postgres-volume: EBUSY: Device or resource busy, retrying in a bit
  2023-12-17 12:50:49.648 UTC [323] LOG:  received smart shutdown request
  2023-12-17 12:50:49.657 UTC [323] LOG:  background worker "logical replication launcher" (PID 329) exited with exit code 1
  2023-12-17 12:50:49.657 UTC [324] LOG:  shutting down
  2023-12-17 12:50:49.657 UTC [324] PANIC:  could not open file "/postgres-volume/run/postgresql/data/global/pg_control": No such file or directory
  2023-12-17 12:50:49.657 UTC [323] LOG:  checkpointer process (PID 324) was terminated by signal 6: Aborted
  2023-12-17 12:50:49.657 UTC [323] LOG:  terminating any other active server processes
  2023-12-17 12:50:49.657 UTC [323] LOG:  abnormal database system shutdown
  2023-12-17 12:50:49.658 UTC [323] LOG:  database system is shut down
   WARN hallpass exited, pid: 314, status: signal: 15 (SIGTERM)
  2023/12/17 12:50:49 listening on [fdaa:3:3969:a7b:1aa:9dfd:12b7:2]:22 (DNS: [fdaa::3]:53)
  [  265.626826] reboot: Restarting system
  [    0.041993] PCI: Fatal: No config space access function found
   INFO Starting init (commit: 15238e9)...
   INFO Mounting /dev/vdb at /postgres-volume w/ uid: 0, gid: 0 and chmod 0755
   INFO Resized /postgres-volume to 1056964608 bytes
   INFO Preparing to run: `docker-entrypoint.sh ./scripts/fly-io-start.sh` as root
   INFO [fly api proxy] listening at /.fly/api
  2023/12/17 12:50:51 listening on [fdaa:3:3969:a7b:1aa:9dfd:12b7:2]:22 (DNS: [fdaa::3]:53)
  Creating folders for PostgreSQL and adding permissions for postgres user...
  PostgreSQL config file exists, starting database...
  waiting for server to start....2023-12-17 12:50:51.405 UTC [323] FATAL:  database files are incompatible with server
  2023-12-17 12:50:51.405 UTC [323] DETAIL:  The data directory was initialized by PostgreSQL version 15, which is not compatible with this version 16.1.
   stopped waiting
  pg_ctl: could not start server
  Examine the log output.
   INFO Main child exited normally with code: 1
✖ Machine 3d8d9299a23278 [app] update failed: smoke checks for 3d8d9299a23278 failed: the app appears to be crashing
   INFO Starting clean up.
   INFO Umounting /dev/vdb from /postgres-volume
   WARN hallpass exited, pid: 314, status: signal: 15 (SIGTERM)
  2023/12/17 12:50:52 listening on [fdaa:3:3969:a7b:1aa:9dfd:12b7:2]:22 (DNS: [fdaa::3]:53)
  [    2.309753] reboot: Restarting system
  machine did not have a restart policy, defaulting to restart
  [    0.041697] PCI: Fatal: No config space access function found
   INFO Starting init (commit: 15238e9)...
   INFO Mounting /dev/vdb at /postgres-volume w/ uid: 0, gid: 0 and chmod 0755
   INFO Resized /postgres-volume to 1056964608 bytes
   INFO Preparing to run: `docker-entrypoint.sh ./scripts/fly-io-start.sh` as root
   INFO [fly api proxy] listening at /.fly/api
  2023/12/17 12:50:54 listening on [fdaa:3:3969:a7b:1aa:9dfd:12b7:2]:22 (DNS: [fdaa::3]:53)
  Creating folders for PostgreSQL and adding permissions for postgres user...
  PostgreSQL config file exists, starting database...
  waiting for server to start....2023-12-17 12:50:54.154 UTC [323] FATAL:  database files are incompatible with server
  2023-12-17 12:50:54.154 UTC [323] DETAIL:  The data directory was initialized by PostgreSQL version 15, which is not compatible with this version 16.1.
   stopped waiting
  pg_ctl: could not start server
  Examine the log output.
   INFO Main child exited normally with code: 1
   INFO Starting clean up.
   INFO Umounting /dev/vdb from /postgres-volume
   WARN hallpass exited, pid: 314, status: signal: 15 (SIGTERM)
  2023/12/17 12:50:55 listening on [fdaa:3:3969:a7b:1aa:9dfd:12b7:2]:22 (DNS: [fdaa::3]:53)
  [    2.323352] reboot: Restarting system
  [    0.040976] PCI: Fatal: No config space access function found
   INFO Starting init (commit: 15238e9)...
   INFO Resized /postgres-volume to 1056964608 bytes
   INFO Preparing to run: `docker-entrypoint.sh ./scripts/fly-io-start.sh` as root
  2023/12/17 12:50:56 listening on [fdaa:3:3969:a7b:1aa:9dfd:12b7:2]:22 (DNS: [fdaa::3]:53)
  PostgreSQL config file exists, starting database...
  waiting for server to start....2023-12-17 12:50:56.934 UTC [323] FATAL:  database files are incompatible with server
  2023-12-17 12:50:56.934 UTC [323] DETAIL:  The data directory was initialized by PostgreSQL version 15, which is not compatible with this version 16.1.
   stopped waiting
  Examine the log output.
Error: smoke checks for 3d8d9299a23278 failed: the app appears to be crashing
Error: Process completed with exit code 1.

cc @ProchaLu @Eprince-hub

karlhorky commented 9 months ago

If we want to address this, we could use a specific major version of PostgreSQL instead of installing the latest one.

karlhorky commented 9 months ago

The easiest way to get around this is to recreate your app / machine / database on Fly.io

As per the guidance above, I tried destroying the app and volume and recreating them, but this led to other No such file or directory problems:

Reverted to PostgreSQL v15 in this commit:

karlhorky commented 8 months ago

PostgreSQL v16 support still needs to be fixed, but I forgot that we're going to do that in this issue:

Closing again.