search

oven-sh / bun

Incredibly fast JavaScript runtime, bundler, test runner, and package manager – all in one
https://bun.sh
Other
74.28k stars 2.77k forks source link

Docker oven/bun:latest fails during bun install #9807

Closed jnsdrssn closed 7 months ago

jnsdrssn commented 7 months ago

What version of Bun is running?

1.1.0

What platform is your computer?

Linux 5.15.0-101-generic aarch64 aarch64

What steps can reproduce the bug?

build a docker file similar to this

FROM oven/bun:latest
WORKDIR /app

# Copy the lock and package file
COPY bun.lockb . 
COPY package.json .
COPY apps/api ./apps/api
COPY packages ./packages

# Install dependencies
RUN bun install

EXPOSE 3001/tcp

USER bun
CMD ["bun", "run", "apps/api/src/index.ts"]

What is the expected behavior?

docker builds successfully

What do you see instead?

0.200 bun install v1.1.0 (5903a614)
6.547 gyp info it worked if it ends with ok
6.547 gyp info using node-gyp@10.1.0
6.547 gyp info using node@21.6.0 | linux | arm64
6.547 gyp ERR! find Python
6.547 gyp ERR! find Python Python is not set from command line or npm configuration
6.547 gyp ERR! find Python Python is not set from environment variable PYTHON
6.547 gyp ERR! find Python checking if "python3" can be used
6.547 gyp ERR! find Python - "python3" is not in PATH or produced an error
6.547 gyp ERR! find Python checking if "python" can be used
6.547 gyp ERR! find Python - "python" is not in PATH or produced an error
6.547 gyp ERR! find Python
6.547 gyp ERR! find Python **********************************************************
6.547 gyp ERR! find Python You need to install the latest version of Python.
6.547 gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
6.547 gyp ERR! find Python you can try one of the following options:
6.547 gyp ERR! find Python - Use the switch --python="/path/to/pythonexecutable"
6.547 gyp ERR! find Python (accepted by both node-gyp and npm)
6.547 gyp ERR! find Python - Set the environment variable PYTHON
6.547 gyp ERR! find Python - Set the npm configuration variable python:
6.547 gyp ERR! find Python npm config set python "/path/to/pythonexecutable"
6.547 gyp ERR! find Python For more information consult the documentation at:
6.547 gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
6.547 gyp ERR! find Python **********************************************************
6.547 gyp ERR! find Python
6.547 gyp ERR! configure error
6.547 gyp ERR! stack Error: Could not find any Python installation to use
6.547 gyp ERR! stack at fail (/tmp/bunx-0-node-gyp@latest/node_modules/node-gyp/lib/find-python.js:305:67)
6.547 gyp ERR! stack at processTicksAndRejections (:12:39)
6.547 gyp ERR! System Linux 5.15.0-101-generic
6.547 gyp ERR! command "/usr/local/bin/bun" "/tmp/bunx-0-node-gyp@latest/node_modules/.bin/node-gyp" "rebuild"
6.547 gyp ERR! cwd /app/node_modules/bufferutil
6.547 gyp ERR! node -v v21.6.0
6.547 gyp ERR! node-gyp -v v10.1.0
6.547 gyp ERR! not ok
6.548
6.548 error: install script from "bufferutil" exited with 1
------
failed to solve: process "/bin/sh -c bun install" did not complete successfully: exit code: 1

Additional information

No response

jonasfroeller commented 7 months ago

updating the system packages worked for me

olinelson commented 7 months ago

updating the system packages worked for me

Hey @jonasfroeller, can you elaborate? How were you able to resolve the issue? Cheers

ShadowLeiPolestar commented 7 months ago

Same issue when upgrading from bun v1.0.35 to v1.1.0 My case: Using GitHub Actions w/ "FROM --platform=linux/amd64 oven/bun:1.1.0-alpine AS BUILD_IMAGE", then when on bun install failed w/ the same issue described.

Interesting thing was that I had 2 projects refers the same action file, however 1 succeed w/ the installment & build, another always met the issue when on "bun install".

Is there anything relates to the image itself? Cache maybe (just guess)?

Succeed one (the image download phase): `#3 [internal] load metadata for docker.io/oven/bun:1.1.0-alpine

3 DONE 1.0s

5 [internal] load build context

5 transferring context: 177.96kB done

5 DONE 0.0s

6 [build_image 1/18] FROM docker.io/oven/bun:1.1.0-alpine@sha256:cd4b6bf06a182b2f79bdd92ebd28b9d95ad1c0939db7e1f7604821f841dbab3a

6 resolve docker.io/oven/bun:1.1.0-alpine@sha256:cd4b6bf06a182b2f79bdd92ebd28b9d95ad1c0939db7e1f7604821f841dbab3a done

6 sha256:1f13e68e59cfc23b02f80f9307b98d6e5991fc7036f9e9e778c48d324c95eb35 0B / 296B 0.1s

6 sha256:f007b4650a0f1b7ebf8544ad1398ad41aa418c679f964bcd526f77dc0dbe41a4 0B / 3.21MB 0.1s

6 sha256:cd4b6bf06a182b2f79bdd92ebd28b9d95ad1c0939db7e1f7604821f841dbab3a 1.61kB / 1.61kB done

6 sha256:87bc6007096733e4276eb93e0dd1e62aa222ee72ba83b9bca2d964d9e270608b 1.24kB / 1.24kB done

6 sha256:e4915e351d9e40f529a217f71eae0db0db754663915fb071fa980e81e69dd86b 3.11kB / 3.11kB done

6 sha256:1f13e68e59cfc23b02f80f9307b98d6e5991fc7036f9e9e778c48d324c95eb35 296B / 296B 0.1s done

6 sha256:f007b4650a0f1b7ebf8544ad1398ad41aa418c679f964bcd526f77dc0dbe41a4 3.21MB / 3.21MB 0.2s done

6 ......

6 DONE 1.7s`

Failed one (the image download phase): `#3 [internal] load metadata for docker.io/oven/bun:1.1.0-alpine

3 DONE 1.0s

5 [internal] load build context

5 transferring context: 243.47kB done

5 DONE 0.0s

6 [build_image 1/18] FROM docker.io/oven/bun:1.1.0-alpine@sha256:cd4b6bf06a182b2f79bdd92ebd28b9d95ad1c0939db7e1f7604821f841dbab3a

6 resolve docker.io/oven/bun:1.1.0-alpine@sha256:cd4b6bf06a182b2f79bdd92ebd28b9d95ad1c0939db7e1f7604821f841dbab3a done

6 sha256:87bc6007096733e4276eb93e0dd1e62aa222ee72ba83b9bca2d964d9e270608b 1.24kB / 1.24kB done

6 sha256:e4915e351d9e40f529a217f71eae0db0db754663915fb071fa980e81e69dd86b 3.11kB / 3.11kB done

6 sha256:98b0fa0d035d5d28241b4e4369bb314a9b556c09fd45df67c4030297045313b6 0B / 36.47MB 0.1s

6 sha256:1f13e68e59cfc23b02f80f9307b98d6e5991fc7036f9e9e778c48d324c95eb35 0B / 296B 0.1s

6 ......

6 DONE 1.6s`

jonasfroeller commented 7 months ago

updating the system packages worked for me

Hey @jonasfroeller, can you elaborate? How were you able to resolve the issue? Cheers

Now it doesn't work anymore 👀. I experienced it similarly to @ShadowLeiPolestar.

Jarred-Sumner commented 7 months ago

From the error message:

6.547 gyp ERR! find Python
6.547 gyp ERR! find Python Python is not set from command line or npm configuration
6.547 gyp ERR! find Python Python is not set from environment variable PYTHON
6.547 gyp ERR! find Python checking if "python3" can be used
6.547 gyp ERR! find Python - "python3" is not in PATH or produced an error
6.547 gyp ERR! find Python checking if "python" can be used
6.547 gyp ERR! find Python - "python" is not in PATH or produced an error
6.547 gyp ERR! find Python
6.547 gyp ERR! find Python **********************************************************
6.547 gyp ERR! find Python You need to install the latest version of Python.
6.547 gyp ERR! find Python Node-gyp should be able to find and use Python. If not,
6.547 gyp ERR! find Python you can try one of the following options:
6.547 gyp ERR! find Python - Use the switch --python="/path/to/pythonexecutable"
6.547 gyp ERR! find Python (accepted by both node-gyp and npm)
6.547 gyp ERR! find Python - Set the environment variable PYTHON
6.547 gyp ERR! find Python - Set the npm configuration variable python:
6.547 gyp ERR! find Python npm config set python "/path/to/pythonexecutable"
6.547 gyp ERR! find Python For more information consult the documentation at:
6.547 gyp ERR! find Python https://github.com/nodejs/node-gyp#installation
6.547 gyp ERR! find Python **********************************************************
6.547 gyp ERR! find Python

Note how node-gyp is complaining about not having Python installed.

In #9875, I added python3 to the list of installed dependencies in the Docker image.

The next issue after this that might come up is a C/C++ compiler.

If you're using ws in Bun, an alternative is to set "trustredDependencies": [] in package.json. That will block running the bufferutil postinstall script, which will skip the node-gyp build step happening here.

tapz commented 7 months ago

I added RUN apk --no-cache add python3 to my Dockerfile and now I'm getting this error:

ERR! install response status 404 Not Found on https://supabase-public-artifacts-bucket.s3.amazonaws.com/libpg-query-node/queryparser-v13.3.2-node-v115-linux-arm64.tar.gz 
17.61 node-pre-gyp WARN Pre-built binaries not installable for libpg-query@13.3.2 and node@21.6.0 (node-v115 ABI, glibc) (falling back to source compile with node-gyp)
...
gyp: name 'llvm_version' is not defined while evaluating condition 'llvm_version=="0.0"' in binding.gyp while trying to load binding.gyp
tapz commented 7 months ago

The problem seems to be with my devDependencies. But as bun install --production does not work when there is no lock file, I need to first run bun install, which tries to install the failing dev dependency (which is not needed in the container). So, the --production should be fixed in bun to generate the lock file.

RUN bun install
RUN bun install --production
tapz commented 7 months ago

As a workaround, I hide the devDependencies:

RUN sed -i "s|devDependencies|bugs |g" ./package.json

thienandangthanh commented 7 months ago

I noticed that there were some changes made to the bun project between versions bun-v1.1.0 and bun-v1.1.1.

On April 4th, python3 was added by @Jarred-Sumner (https://github.com/oven-sh/bun/commit/76795af6954c058655b6a020a079df94ce917087). However, quickly after that, python3 was removed by @Electroid (https://github.com/oven-sh/bun/commit/0dfbdc711ae3e87e1e683940e4708fe81b8adaa7).

Could you please provide some context for these changes? It would be helpful to understand the reasoning behind the addition and subsequent removal of python3.

Thank you for your time and contributions to this project.

Jarred-Sumner commented 7 months ago

@thienandangthanh I added python3 to all the Docker images because the error message says to add python3

@Electroid removed python3 from the alpine & slim builds because it increases image size and the equivalent Node.js docker images do not add python3

thienandangthanh commented 7 months ago

@Jarred-Sumner Thank you for your quick response and for clarifying the changes made to the bun project. I understand now that python3 was added to address an error message, and later removed from the alpine & slim builds to maintain a smaller image size. Your explanation makes perfect sense.

I appreciate your efforts in maintaining the efficiency and effectiveness of this project. Keep up the great work!

thienandangthanh commented 7 months ago

@jnsdrssn

I've encountered a similar error to the one reported in this issue. The error seems to be related to the sharp and sharp-libvips packages. I found a related issue here: https://github.com/oven-sh/bun/issues/267#issuecomment-2038309803.

Interestingly, when I use the Docker image oven/bun:1.0.31-alpine, the problem does not occur. This might suggest that the issue is environment-specific.

Here's the error log with bun 1.1.2 for reference:

RUN bun install --frozen-lockfile:
0.301 bun install v1.1.2 (c8d072c2)
136.0 sharp: Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.14.4/libvips-8.14.4-linux-x64.tar.br
...
136.0 error: zlib.BrotliDecompress is not implemented
...
136.0 gyp ERR! find Python
...
136.0 gyp ERR! find Python **********************************************************
136.0 gyp ERR! find Python You need to install the latest version of Python.
...
136.0 gyp ERR! find Python **********************************************************
...
136.0 gyp ERR! configure error
136.0 gyp ERR! stack Error: Could not find any Python installation to use
...
136.0 gyp ERR! not ok
136.0
136.0 error: install script from "sharp" exited with 1

The error message suggests that zlib.BrotliDecompress is not implemented, and there seems to be an issue with finding a Python installation.

I hope this additional information can assist the Bun team in diagnosing and resolving the issue. Please let me know if there's any other information I can provide to help.

jaredLunde commented 7 months ago

To workaround the issue I've been using node:lts-slim as my base image. 

FROM --platform=$BUILDPLATFORM node:lts-slim AS base
WORKDIR /app

COPY . .
RUN npm i -g bun

FROM base AS all-deps
RUN bun install

FROM base AS prod-deps
RUN bun install --production

FROM all-deps AS build
ENV NODE_ENV=production
RUN bun --bun run astro build

FROM oven/bun:alpine AS runtime
COPY --from=build /app/dist dist
COPY --from=prod-deps /app/node_modules node_modules
coindegen commented 7 months ago

To workaround the issue I've been using node:lts-slim as my base image.

Thanks for the suggestion @jaredLunde, switching my builder image from oven/bun to node:lts successfully resolved my bun install errors.

I was getting the same errors as @tapz above while installing bufferutil:

22.79 gyp: name 'llvm_version' is not defined while evaluating condition 'llvm_version=="0.0"' in binding.gyp while trying to load binding.gyp

Explicitly installing llvm and setting ENV llvm_version=17 explicitly didn't help. Setting "trustedDependencies": [] in package.json didn't fix it either.

In my case though bufferutil was part of a regular dependency—not a dev-dependency—so I couldn't just skip installing dev dependencies. The prior workaround was to pass the --ignore-scripts flag like this: bun install --ignore-scripts. I prefer not having to skip those though.

smaccoun commented 7 months ago

Unfortunately I'm getting the exact same python error as when using the bun base image for this suggestion

To workaround the issue I've been using node:lts-slim as my base image.

FROM --platform=$BUILDPLATFORM node:lts-slim AS base
WORKDIR /app

COPY . .
RUN npm i -g bun

FROM base AS all-deps
RUN bun install

FROM base AS prod-deps
RUN bun install --production

FROM all-deps AS build
ENV NODE_ENV=production
RUN bun --bun run astro build

FROM oven/bun:alpine AS runtime
COPY --from=build /app/dist dist
COPY --from=prod-deps /app/node_modules node_modules
"python3" is not in PATH or produced an error
thienandangthanh commented 7 months ago

@smaccoun Have you tried standard image like node:lts? As Jarred-Sumner stated that alpine and slim Node.js images don't include python3 https://github.com/oven-sh/bun/issues/9807#issuecomment-2038823015

smaccoun commented 7 months ago

@smaccoun Have you tried standard image like node:lts? As Jarred-Sumner stated that alpine and slim Node.js images don't include python3 #9807 (comment)

Ah yes I didn't read carefully enough. Thanks for pointing that out @thienandangthanh. I do have it working now

I'm a bit confused by this comment @Jarred-Sumner

@thienandangthanh I added python3 to all the Docker images because the error message says to add python3

@Electroid removed python3 from the alpine & slim builds because it increases image size and the equivalent Node.js docker images do not add python3

Seems like a critical dependency to make the docker image work with bun build, so why not include it? I supposed one reason might be that some might just be using bun for it's runtime, and not to actually build? That would make sense if the reason. I wonder though about then maybe have two official Dockerfiles (bun-run and bun-build) and bun build would be setup with all the required dependencies, including python in this case.

Also just my 2 cents here, but part of what I love about Bun ux is things seem to just work. I don't have to spend tons of time configuring things. Building your backend on docker is going to be a very common use case. Yet here i have to say I feel like there is an odd (almost universal) developer obsession with container sizes, bundle sizes, etc. In many instances it seems like we forego good dx to save some size that will have 0 perceivable downstream impact. To me it's effectively a classic case of premature optimization. I understand the general sentiment to not make a container any larger than it has to be, and to leave it up to the user to add what they need, but in this case how much does Python3 really affect anyone's setup? I'm not strongly opinionated on this, just something to consider. I get many might object to this, so there's probably no clear answer and I'd understand whatever direction you decide to go. I certainly don't regard @Electroid decision to remove it as a bad decision by any stretch - it may be the popular thing to do amongst many developers and therefore the right thing from a bun perspective. I guess I just sometimes want to push back against the whole developer community and the seemingly endless desire to squeeze size out of everything, even when it seems like the tradeoffs are quite lopsided

thienandangthanh commented 6 months ago

I've encountered a similar error to the one reported in this issue. The error seems to be related to the sharp and sharp-libvips packages. I found a related issue here: #267 (comment).

Interestingly, when I use the Docker image oven/bun:1.0.31-alpine, the problem does not occur. This might suggest that the issue is environment-specific.

Here's the error log with bun 1.1.2 for reference:

RUN bun install --frozen-lockfile:
0.301 bun install v1.1.2 (c8d072c2)
136.0 sharp: Downloading https://github.com/lovell/sharp-libvips/releases/download/v8.14.4/libvips-8.14.4-linux-x64.tar.br
...
136.0 error: zlib.BrotliDecompress is not implemented
...
136.0 gyp ERR! find Python
...
136.0 gyp ERR! find Python **********************************************************
136.0 gyp ERR! find Python You need to install the latest version of Python.
...
136.0 gyp ERR! find Python **********************************************************
...
136.0 gyp ERR! configure error
136.0 gyp ERR! stack Error: Could not find any Python installation to use
...
136.0 gyp ERR! not ok
136.0
136.0 error: install script from "sharp" exited with 1

The error message suggests that zlib.BrotliDecompress is not implemented, and there seems to be an issue with finding a Python installation.

I hope this additional information can assist the Bun team in diagnosing and resolving the issue. Please let me know if there's any other information I can provide to help.

With the bun 1.1.8 release, my problem is gone. Thanks to node:zlib brotli support in pull request https://github.com/oven-sh/bun/pull/10722.

Now I can use the Docker image oven/bun:1.1.8-alpine without the problem.

skusez commented 4 months ago

Still seems to be an issue

I had to downgrade to 1.0.x to get it working:

FROM --platform=linux/amd64 oven/bun:1.0.31-alpine AS base
tomholford commented 4 months ago

For those using the default image (based on debian), this worked as a temporary workaround for the missing python3 err:

RUN apt update && apt install python3 python3-pip make g++ -y
oDestroyeRo commented 4 weeks ago

For alpine

RUN apk --no-cache add python3 gcc make