search

awslabs / aws-crt-nodejs

NodeJS bindings for the AWS Common Runtime.
Apache License 2.0
37 stars 24 forks source link

yarn install for aws-crt-nodejs: CMake Error: CMAKE_C_COMPILER not set #498

Closed ningji closed 7 months ago

ningji commented 8 months ago

Describe the bug

Running docker build from MacOS,

FROM artifactory.kbra.com/docker/node:18.3.0-alpine3.16
...
RUN apk add --no-cache -qq libzip-dev gnupg yarn pcre-dev g++ cmake bash > /dev/null
RUN yarn install --production --ignore-optional > /dev/null

in package.json

  "dependencies": {
    "@aws-sdk/client-s3": "3.437.0",
    "@aws-sdk/client-sqs": "3.437.0",
    "@aws-sdk/signature-v4-crt": "3.437.0",
    "@azure/storage-blob": "12.16.0",
    "@azure/storage-file-share": "12.16.0",
    "cmake-js": "6.3.2",
    "tar": "6.1.11",
    ......
  }

The node version in the docker container is 18.

The errors:

213.3 npm WARN config global `--global`, `--local` are deprecated. Use `--location=global` instead.
213.3 Could not find cmake-js version 6.3.2

213.3 Not searching for unused variables given on the command line.
213.3 CMake Error: CMake was unable to find a build program corresponding to "Unix Makefiles".  CMAKE_MAKE_PROGRAM is not set.  You probably need to select a different build tool.
213.3 CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage

However https://www.npmjs.com/package/cmake-js?activeTab=versions has cmake-js

Expected Behavior

expects aws-crt can be installed

Current Behavior

213.3 error /app/node_modules/aws-crt: Command failed. 213.3 Exit code: 1 213.3 Command: node ./scripts/install.js 213.3 Arguments: 213.3 Directory: /app/node_modules/aws-crt 213.3 Output: 213.3 npm WARN config global --global, --local are deprecated. Use --location=global instead. 213.3 Could not find tar version 6.1.11 213.3 Looking for tar version 6.1.11 as a dependency... 213.3 npm WARN config global --global, --local are deprecated. Use --location=global instead. 213.3 Could not find tar version 6.1.11 213.3 Could not find tar 213.3 Downloading tar from npm for build... 213.3 npm notice 213.3 npm notice New major version of npm available! 8.11.0 -> 10.2.1 213.3 npm notice Changelog: https://github.com/npm/cli/releases/tag/v10.2.1 213.3 npm notice Run npm install -g npm@10.2.1 to update! 213.3 npm notice 213.3 Found axios in node list! 213.3 Source file download succeed! 213.3 npm WARN config global --global, --local are deprecated. Use --location=global instead. 213.3 Could not find cmake-js version 6.3.2 213.3 Looking for cmake-js version 6.3.2 as a dependency... 213.3 npm WARN config global --global, --local are deprecated. Use --location=global instead. 213.3 Could not find cmake-js version 6.3.2 213.3 Could not find cmake-js 213.3 Downloading cmake-js from npm for build... 213.3 info TOOL Using Unix Makefiles generator. 213.3 info TOOL Building only the install target, as specified from the command line. 213.3 info DIST Downloading distribution files to: /root/.cmake-js/node-arm64/v18.3.0 213.3 http DIST - https://nodejs.org/dist/v18.3.0/SHASUMS256.txt 213.3 http DIST - https://nodejs.org/dist/v18.3.0/node-v18.3.0-headers.tar.gz 213.3 info CMD CONFIGURE 213.3 info TOOL Building only the install target, as specified from the command line. 213.3 info RUN [ 213.3 info RUN 'cmake', 213.3 info RUN '/app/node_modules/aws-crt', 213.3 info RUN '--no-warn-unused-cli', 213.3 info RUN '-G', 213.3 info RUN 'Unix Makefiles', 213.3 info RUN '-DCMAKE_JS_VERSION=6.3.2', 213.3 info RUN '-DCMAKE_BUILD_TYPE=Release', 213.3 info RUN '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/app/node_modules/aws-crt/build/linux-arm64-musl/Release', 213.3 info RUN '-DCMAKE_JS_INC=/root/.cmake-js/node-arm64/v18.3.0/include/node', 213.3 info RUN '-DCMAKE_JS_SRC=', 213.3 info RUN '-DNODE_RUNTIME=node', 213.3 info RUN '-DNODE_RUNTIMEVERSION=18.3.0', 213.3 info RUN '-DNODE_ARCH=arm64', 213.3 info RUN '-DCMAKE_EXPORT_COMPILE_COMMANDS=true', 213.3 info RUN '-DCMAKE_JS_PLATFORM=linux', 213.3 info RUN '-DBUILD_TESTING=OFF', 213.3 info RUN '-DCMAKE_INSTALL_PREFIX=crt/install', 213.3 info RUN '-DCMAKE_PREFIX_PATH=crt/install' 213.3 info RUN ] 213.3 Not searching for unused variables given on the command line. 213.3 CMake Error: CMake was unable to find a build program corresponding to "Unix Makefiles". CMAKE_MAKE_PROGRAM is not set. You probably need to select a different build tool. 213.3 CMake Error: CMAKE_C_COMPILER not set, after EnableLanguage 213.3 -- Configuring incomplete, errors occurred! 213.3 See also "/app/node_modules/aws-crt/build/linux-arm64-musl/CMakeFiles/CMakeOutput.log". 213.3 ERR! OMG Process terminated: 1 213.3 tar perform step failed withError: Process terminated: 1 213.3 node:internal/process/promises:288 213.3 triggerUncaughtException(err, true / fromPromise /); 213.3 ^ 213.3 213.3 Error: Process terminated: 1 213.3 at ChildProcess. (/app/node_modules/cmake-js/lib/processHelpers.js:34:32) 213.3 at ChildProcess.emit (node:events:527:28) 213.3 at ChildProcess._handle.onexit (node:internal/child_process:291:12) 213.3 213.3 Node.js v18.3.0

Reproduction Steps

dockerfile

FROM artifactory.kbra.com/docker/node:18.3.0-alpine3.16
...
RUN apk add --no-cache -qq libzip-dev gnupg yarn pcre-dev g++ cmake bash > /dev/null
RUN yarn install --production --ignore-optional > /dev/null

in package.json


  "dependencies": {
    "@aws-sdk/client-s3": "3.437.0",
    "@aws-sdk/client-sqs": "3.437.0",
    "@aws-sdk/signature-v4-crt": "3.437.0",
    "@azure/storage-blob": "12.16.0",
    "@azure/storage-file-share": "12.16.0",
    "cmake-js": "6.3.2",
    "tar": "6.1.11",
    ......
  }

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### aws-crt-nodejs version used

1.18.3

### nodejs version used

18.3.0

### Operating System and version

macOs
ningji commented 8 months ago

more details:

this is the old working versions with an existing yarn.lock (it would have similar issue if i regenerate this yarn.lock)

├─┬ @aws-sdk/client-s3@3.258.0 │ └─┬ @aws-sdk/util-user-agent-node@3.257.0 │ └── aws-crt@1.15.1 deduped ├─┬ @aws-sdk/client-sqs@3.391.0 │ └─┬ @aws-sdk/util-user-agent-node@3.391.0 │ └── aws-crt@1.15.1 deduped └─┬ @aws-sdk/signature-v4-crt@3.257.0 └── aws-crt@1.15.1

ningji commented 8 months ago

i can install everything properly in Mac with the same package.json, same node 18. It's just failing in docker.

bretambrose commented 8 months ago

Can you include your full docker container definition?

ningji commented 8 months ago

@bretambrose Thx for getting back to me.

1st of all i'm still confused when is the 1st time this problem shows up, the docker build works for an old versions of "@aws-sdk/signature-v4-crt": "3.257.0" with a special yarn.lock, but not a refresh built yarn.lock (delete it 1st, then run yarn install)

FROM artifactory.myCompany.com/docker/node:16.18.1-alpine3.16
WORKDIR /app/

RUN apk add --no-cache -qq libzip-dev gnupg yarn pcre-dev g++ cmake bash > /dev/null

COPY . /app

RUN yarn install --production --ignore-optional > /dev/null

i'm pretty sure node:16.18.1-alpine3.16 is the same one from official site, my workplace doesn't make any changes

bretambrose commented 8 months ago

I will take a look when I can, but it will probably not be until Wednesday (11/1). I do believe we recently started publishing an alpine binary (x64 only) in the JS CRT, but no idea if that's related or not.

ningji commented 8 months ago

Thx a lot, take your time. i can live with the existing package.json and yarn.lock for a while. If i found anything new, i'll update here. Best regards !

sacha-c commented 7 months ago

I also ran into this issue, and as suggested by the error CMake was unable to find a build program corresponding to "Unix Makefiles" installing make with apk add make resolved it for me

ningji commented 7 months ago

I also ran into this issue, and as suggested by the error CMake was unable to find a build program corresponding to "Unix Makefiles" installing make with apk add make resolved it for me

Hi Sacha, could u pls post more details, e.g. the line of "apk add" in the dockerfile ? Adding "make" helps, but i continue get these issues:

22.94 gyp ERR! find Python Python is not set from command line or npm configuration
...
99.41 Could not find tar version 6.1.11
...

Many thanks !

sacha-c commented 7 months ago

Looks like it fixed your original issue, but now you are running into a different one

Could you post more of the error logs?
Are you using the same Dockerfile as in your previous comment here?
Can you share your package.json to see what other dependencies you have which could cause this issue?

If it helps, my setup looks like this:

FROM node:18.8-alpine

# Build tools for `aws-crt`
RUN apk add g++ cmake make

COPY package.json yarn.lock ./

RUN yarn install --check-files --frozen-lockfile --no-progress

package.json

{
  ...
  },
  "dependencies": {
    ...
    "aws-crt": "^1.18.3",
ningji commented 7 months ago

Looks like it fixed your original issue, but now you are running into a different one

Could you post more of the error logs? Are you using the same Dockerfile as in your previous comment here? Can you share your package.json to see what other dependencies you have which could cause this issue?

Thx very much for the details, i'm stuck on sth. else and will come back for this some time later, best regards !

github-actions[bot] commented 7 months ago

Greetings! It looks like this issue hasn’t been active in longer than a week. We encourage you to check if this is still an issue in the latest release. Because it has been longer than a week since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment or add an upvote to prevent automatic closure, or if the issue is already closed, please feel free to open a new one.