Closed skorfmann closed 2 years ago
@eladb We don't want to have bundled assets owned by root (see #8489) but at the same time we should allow running the container as root...
Need to try this but how about running the image again with command: ['sh', '-c', 'chmod -R 777 /asset-output']
after this (both runs would run as root)?
@jogold not sure how setting all files to 777 under /asset-output would solve this problem
@jogold not sure how setting all files to 777 under /asset-output would solve this problem
If we go back to running the container as root we will have content owned by root in /asset-output
(#8489). A solution could be to run as root and then run again as root to change permissions (hacky?).
Oh, men! I would have saved a lot of time with the fix proposed by @jogold
After enabling debug in docker daemon, running lambda docker images by hand, I figured this.
At least in a Python project using cdk 1.47.1, chmod -R 777 .cdk.staging
after cdk synth --staging
allowed me to run successfully 'sam local invoke' a lambda bundling asset code
I am getting similar error but for go-1-x lambda
Failed to run bundling Docker image for asset xxx Error: spawnSync docker ENOENT
I am getting similar error but for go-1-x lambda
Failed to run bundling Docker image for asset xxx Error: spawnSync docker ENOENT
This error message means that docker
is not available. You can make docker
available in your environment, use the CDK_DOCKER
env var or use the new "local bundling" alternative (the latest will be available in the next release)
This error message means that
docker
is not available. You can makedocker
available in your environment, use theCDK_DOCKER
env var
@jogold I don't understand what you mean by this. The environment variable takes key and value. You have mentioned only 1 of them(CDK_DOCKER
). And my guess is CDK_DOCKER
is the key and its value is missing. I am new to this and was trying to follow building go lambda inside docker tutorial.
or use the new "local bundling" alternative (the latest will be available in the next release)
The alternative you suggested has very little explanation and no sample code. But I am guessing you are recommending building lambda on my local machine. Which I already do with Taskfile and deploy zip file. This solution has been working for me.
Between local bundling and docker bundling I feel docker solution is better solution. So I would like to figure out how to make it work. So if you could provide sample code for golang lambda then it would be great.
Docker was working with Cloud9 editor but I didn't really like Cloud9 and I moved to using VS Code and sshing into EC2 instance. Since then it stopped working. I guess I need that CDK_DOCKER
env variable that is missing.
Can you check if docker
is available/installed in your environment and in your PATH
? Are you able to run docker
in your EC2 instance?
@jogold No its not installed/available. But there is special docker version for AWS right. This is directly going to cloudformation template. Is there CDK way to install it instead? Or should I just install regular Ubuntu version instead? I am running Ubuntu 20.04 on EC2
Yes you have to have Docker installed, so in your case Docker for Ubuntu
@jogold Thanks for patiently helping me out. I don't know what I did. I copy pasted lot of code from lot of google search. I don't know what it means but the net result is I have successfully build lambda inside docker container and deployed to aws. I tested api to verify if lambda is actually been deployed. Also after deploy I tried cdk diff
and it actually printed There were no differences
. It does bundling each time I do cdk diff
but also knows if code has changed or not.
I suppose that means this issue is resolved. Thanks for helping out @jogold.
I suppose that means this issue is resolved. Thanks for helping out @jogold.
@nija-at I don't think because @apoorvmote issue wasn't related to the original issue, created by bundling assets owned by root according to @jogold
Hi, i am a newby. Please be patient with me. The following bundling command worked for my lambda functions.
const command = [
'bash', '-c',
`cp -aur . /asset-output &&
cd /asset-output &&
mkdir .npm &&
export npm_config_cache=.npm &&
npm install`,
];
I changed the cache location to a folder created by the current user.
The missing thing for me was it needs to be run by root.
const myLayer = new LayerVersion(this, 'my-layer', {
code: Code.fromAsset(assetPath, {
assetHashType: cdk.AssetHashType.BUNDLE,
bundling: {
user: 'root',
...
I am back at the same issue. 1) I switched from intel computer to arm computer. And docker has some issue running intel container inside arm computer. So its docker issue not cdk issue. 2) I switched from golang lambda to nodejs lambda so now this issue interests me.
@skorfmann This is because npm
wants to write cache in root folder, to fix this:
const layer = new lambda.LayerVersion(this, 'MyLayer', {
code: lambda.Code.fromAsset(path.join(__dirname, '../lambda/'), {
bundling: {
image: lambda.Runtime.NODEJS_12_X.bundlingDockerImage,
command: [
'bash', '-xc', [
'export npm_config_update_notifier=false', // Disable npm upgrade check
'export npm_config_cache=$(mktemp -d)', // Change npm default cache folder
'cd $(mktemp -d)',
'cp -v /asset-input/package*.json .',
'npm i --only=prod',
'mkdir -p /asset-output/nodejs/',
'cp -au node_modules /asset-output/nodejs/',
].join('&&'),
],
},
}),
compatibleRuntimes: [lambda.Runtime.NODEJS_12_X],
description: 'A layer to test the L2 construct',
});
@skorfmann This is because
npm
wants to write cache in root folder, to fix this:
Thanks, this got it working for me. Because it seems happy to create the directory itself and each run will get a fresh container we can just pass those options as enviroment variables:
code: lambda.Code.fromAsset(path.join(__dirname, "../src/"), {
bundling: {
image: lambda.Runtime.NODEJS_14_X.bundlingImage,
environment: {
npm_config_cache: "/tmp/npm_cache",
npm_config_update_notifier: "false",
},
command: [
"sh",
"-c",
[
"cd $(mktemp -d)",
"cp /asset-input/* .",
"npm ci",
"npm run build",
"npm ci --production",
"cp -r ./node_modules ./dist/* /asset-output/",
].join(" && "),
],
},
}),
Maybe we can resolve this issue by having those set by default in the bundling docker image? It would also be nicer if the default user had slightly broader permissions to get around the mktemp, I wasn't able to have it work by specifying /tmp/build-dir as the workingDir parameter.
I'm going to close this since there is a solution. Specifically for bundling nodejs based lambda functions you can use the aws-lambda-nodejs
which automatically handles this.
The assets bundling functionality is meant to be very generalized so I don't think we would accept functionality that is specific to an individual tool. Those types of cases can be handled with higher level constructs (i.e. aws-lambda-nodejs
).
Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.
what is the cause of the error ?
When using the
bundling
option to provide Lambda code, the default behaviour for NODEJS based runtimes is broken, since it's using a non-root user within the Docker container. It can be manually fixed by providingroot
as user as part of thebundling
options.Reproduction Steps
Error Log
Environment
Other
Since the fix is passing
user: 'root'
as argument, I think this change is causing the behaviour https://github.com/aws/aws-cdk/pull/8492This is :bug: Bug Report