[aws-lambda-nodejs] .parcel-cache permission denied in BitBucket Pipelines

[SergKam]

Bitbucket Pipeline failed to create NodejsFunction in docker. When I attempt to create a lambda using that construct and synthesize, I get an error [Error: EACCES: permission denied, mkdir '/asset-input/.parcel-cache']

Reproduction Steps

import * as cdk from '@aws-cdk/core';
import * as nodejs from '@aws-cdk/aws-lambda-nodejs';

export class BugStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id, props);

    new nodejs.NodejsFunction(this, 'handler');
  }
}

The handler a bit complex to show here, but it is irrelevant.

Error Log

> zolar-cdk@1.0.0 deploy /opt/atlassian/pipelines/agent/build/.cdk
> cdk --trace deploy "-a" "projects/sfbridge/create.js"
Bundling asset dev-serg-test-sfbridge/dev-serg-test-sfbridge-sf-zpc-bridge/Code/Stage...
[Error: EACCES: permission denied, mkdir '/asset-input/.parcel-cache'] {
  errno: -13,
  code: 'EACCES',
  syscall: 'mkdir',
  path: '/asset-input/.parcel-cache'
}
/opt/atlassian/pipelines/agent/build/.cdk/node_modules/@aws-cdk/core/lib/asset-staging.js:123
            throw new Error(`Failed to run bundling Docker image for asset ${this.node.path}: ${err}`);
            ^
Error: Failed to run bundling Docker image for asset dev-serg-test-sfbridge/dev-serg-test-sfbridge-sf-zpc-bridge/Code/Stage: Error: docker exited with status 1
    at AssetStaging.bundle (/opt/atlassian/pipelines/agent/build/.cdk/node_modules/@aws-cdk/core/lib/asset-staging.js:123:19)
    at new AssetStaging (/opt/atlassian/pipelines/agent/build/.cdk/node_modules/@aws-cdk/core/lib/asset-staging.js:38:35)
    at new Asset (/opt/atlassian/pipelines/agent/build/.cdk/node_modules/@aws-cdk/aws-s3-assets/lib/asset.js:21:25)
    at AssetCode.bind (/opt/atlassian/pipelines/agent/build/.cdk/node_modules/@aws-cdk/aws-lambda/lib/code.js:137:26)
    at new Function (/opt/atlassian/pipelines/agent/build/.cdk/node_modules/@aws-cdk/aws-lambda/lib/function.js:80:33)
    at new NodejsFunction (/opt/atlassian/pipelines/agent/build/.cdk/node_modules/@aws-cdk/aws-lambda-nodejs/lib/function.js:30:13)
    at ZolarSFBridge.scheduleLambda (/opt/atlassian/pipelines/agent/build/.cdk/projects/sfbridge/ZolarSFBridge.js:34:24)
    at new ZolarSFBridge (/opt/atlassian/pipelines/agent/build/.cdk/projects/sfbridge/ZolarSFBridge.js:29:14)
    at Object.<anonymous> (/opt/atlassian/pipelines/agent/build/.cdk/projects/sfbridge/create.js:7:19)
    at Module._compile (internal/modules/cjs/loader.js:1133:30)
Subprocess exited with error 1

Environment

• CLI Version : 1.53.0
• Framework Version: 1.53.0
• Node.js Version: 12
• OS : lambci/lambda:build-nodejs12.x
• Language (Version): TypeScript (3.8.3)

Other

looks related to #8757

---

This is :bug: Bug Report

[jogold]

Hi @SergKam,

Have you tried playing with the cacheDir option?

[SergKam]

@jogold No, but I found a workaround for now. I'm creating directory .parcel-cache with 0777 permissions in the current working directory before running the CDK. I think it doesn't matter what cacheDir I set they all should be inside my working dir (restriction by BitBacket docker daemon). To make it work "out of the box" cdk should create this directory with the right permissions before running parcel in docker.

[anilchinnam]

I am also running into this issue and unable to upgrade my cdk version from 1.36.1 to 1.58.0. I tried couple of previous versions to upgrade but the issue is been there for quite some time.

Please some one help, what is the work around on this, I tried providing the 777 permission to both .cdk.staging and .parcel-cache folders and deleted these folders before running the synth. Still getting the same error. This is on my Mac and i have the docker running (not sure why it is looking for docker in first place for bundling)

Bundling asset hello-world-dev-ApiDynamoDbStack/dlqS3UtilFunction/dlqS3UtilFunction/Code/Stage... [Error: EACCES: permission denied, mkdir '/.parcel-cache'] { errno: -13, code: 'EACCES', syscall: 'mkdir', path: '/.parcel-cache' } Error: Failed to bundle asset hello-world-dev-ApiDynamoDbStack/dlqS3UtilFunction/dlqS3UtilFunction/Code/Stage: Error: docker exited with status 1 at AssetStaging.bundle (/Users/achinnam/Applications/hello-world/hello-world-app/node_modules/@aws-cdk/core/lib/asset-staging.ts:196:13) at new AssetStaging (/Users/achinnam/Applications/hello-world/hello-world-app/node_modules/@aws-cdk/core/lib/asset-staging.ts:93:29)

[SergKam]

@anilchinnam In my case, I'm using docker to build and deploy lambdas with cdk. my Dockerfile looks like this

FROM lambci/lambda:build-nodejs12.x
WORKDIR /cdk/
COPY bin bin
COPY config config
COPY lib lib
COPY projects projects
COPY index.ts .
COPY package.json .
COPY package-lock.json .
COPY tsconfig.json .
COPY cdk.json .
ENV NODE_ENV production
# Parcel need this to find project root for NodeJsLambda
RUN mkdir --parents .git/keep && mkdir --mode=0777 .parcel-cache && chown 1000:1000 .parcel-cache && \
    npm i && npm run build && npm run build:lambdas
WORKDIR /app/

aws cdk uses another docker image with "parcel" inside to build lambda, and inside this "parcel docker" it uses a user with user id 1000. so i added chown 1000:1000 .parcel-cache just to be sure. I don't know is it actually needed since mode 777 allows write to everyone.

ps: my setup only works up to cdk version 1.54 (i cannot upgrade because another issue https://github.com/aws/aws-cdk/issues/9351)

[anilchinnam]

@SergKam Thank you for the response. Our app currently not using docker with the CDK 1.36.1 version. This is one of the main reason we could not upgrade past 1.36.1 as nodejs-lambda bundle functionality is changed since 1.37.0 . I do not see if there is any AWS documentation on how to organize our code base for the lambda functions in order to build with the docker file.

Any input on the folder structure or documentation for building multiple lambdas in a project is helpful for us to upgrade until 1.54 version. Thanks Again.

[mbonig]

So, not sure if it helps anyone else, but I found I was having this similar issue. I had a direct "dependency" in my package.json to parcel and parcel-bundler. Removing those (and a rm -rf node_modules && npm i) resolved the issue for me.

Hope that helps someone else.

[beanworks-eng-shared]

☝️ This helped me fix my problem as well. To be clear I have to run npm uninstall parcel to remove parcel from my packages.json But make sure you have parcel installed globally: npm install -g parcel@next

[rv-rmiranda]

I am using AWS CDK 1.63.0, and Docker 9.03.8, in my Mac I have no issue creating the bundles and deploy the Lambdas. But in CircleCI I am getting the error:

[Error: EACCES: permission denied, mkdir '/asset-input/src/lambdas/proxy/.parcel-cache'] {
  errno: -13,
  code: 'EACCES',
  syscall: 'mkdir',
  path: '/asset-input/src/lambdas/proxy/.parcel-cache'
}

Also, I have no cache, so I am installing all node modules on each deployment, so I don't think is an NPM Dependenciesissue in my case. Any ideas on how to fix this?

[jogold]

@rv-rmiranda can you share your folder structure and also how you're instantiating the NodejsFunction construct?

[jogold]

@eladb I wonder if we should always run docker bundling as root when in CI (process.env.CI). Not only for aws-lambda-nodejs but in core:

We switched to a non root user only to avoid creating content that is owned by root (see https://github.com/aws/aws-cdk/issues/8489, https://github.com/aws/aws-cdk/pull/8492). But in CI we don't care about this, no?

[eladb]

I wonder if we should always run docker bundling as root when in CI

What's the reason /asset-input/src/lambdas/proxy/.parcel-cache is not writable by the CI user?

[rv-rmiranda]

@rv-rmiranda can you share your folder structure and also how you're instantiating the NodejsFunction construct?

Folder Structure

./infrastructure
 |— bin/
 |———— type_script_infrastructure.ts
 |— lib/
 |———— proxy-stack.ts
 |— cdk.json
 |— package.json
 |— ...
./src
|—— lambdas/
|———— proxy/
|—————— proxy.js
|—————— package.json

Lambda Stack:

import * as core from "@aws-cdk/core";
import { Runtime } from "@aws-cdk/aws-lambda";
import { NodejsFunction } from "@aws-cdk/aws-lambda-nodejs";

export class ProxyStack extends core.Stack {
  public proxy: NodejsFunction;

  constructor(scope: core.App, id: string, props?: core.StackProps) {
    super(scope, id, props);

    const stage = process.env.STAGE || "development";

    this.proxy = new NodejsFunction(this, "LambdaProxy", {
      entry: "../src/lambdas/proxy/proxy.js",
      functionName: `lambda-name-${stage}`,
      handler: "handler",
      memorySize: 1048,
      runtime: Runtime.NODEJS_12_X,
      timeout: core.Duration.minutes(15),
      minify: (stage === "production")? true : false,
    });
  }
}

Please, let me know if you need anything else.

Thank you!

[jogold]

Please, let me know if you need anything else.

Are you using the docker executor or machine executor? https://support.circleci.com/hc/en-us/articles/360007324514-How-can-I-use-Docker-volume-mounting-on-CircleCI and https://circleci.com/docs/2.0/executor-types/#using-machine

[rv-rmiranda]

Please, let me know if you need anything else.

Are you using the docker executor or machine executor? https://support.circleci.com/hc/en-us/articles/360007324514-How-can-I-use-Docker-volume-mounting-on-CircleCI and https://circleci.com/docs/2.0/executor-types/#using-machine

@jogold — I was using docker executor, I change it to machine executor, and now is working! 🎉 🍻

Working Code

version: 2.1
jobs:
  build_and_deploy:
    machine:
      image: ubuntu-1604:202007-01
    working_directory: ~/repo
    steps:
      - checkout
      - attach_workspace:
          at: .

... MORE CODE ...

Not Working Code

version: 2.1
jobs:
  build_and_deploy:
    docker:
      - image: cimg/python:3.8.4
    working_directory: ~/repo
    steps:
      - checkout
      - attach_workspace:
          at: .

      - setup_remote_docker:
          version: 19.03.12
          docker_layer_caching: true

... MORE CODE ...

Thank you for the help!

[eladb]

Please, let me know if you need anything else.

Are you using the docker executor or machine executor? https://support.circleci.com/hc/en-us/articles/360007324514-How-can-I-use-Docker-volume-mounting-on-CircleCI and https://circleci.com/docs/2.0/executor-types/#using-machine

@jogold — I was using docker executor, I change it to machine executor, and now is working! 🎉 🍻

Working Code

version: 2.1

jobs:

  build_and_deploy:

    machine:

      image: ubuntu-1604:202007-01

    working_directory: ~/repo

    steps:

      - checkout

      - attach_workspace:

          at: .

... MORE CODE ...

Not Working Code

version: 2.1

jobs:

  build_and_deploy:

    docker:

      - image: cimg/python:3.8.4

    working_directory: ~/repo

    steps:

      - checkout

      - attach_workspace:

          at: .

      - setup_remote_docker:

          version: 19.03.12

          docker_layer_caching: true

... MORE CODE ...

Thank you for the help!

Should we add a note in our docs?

[jogold]

@eladb yes, will do.

@SergKam were you able to find a solution by changing your pipeline configuration (similar to the CircleCI solution) or do you still rely on creating the folder beforehand?

[SergKam]

@jogold As far as I know, there is no option in bitbucket pipelines to switch to anything else except Docker https://support.atlassian.com/bitbucket-cloud/docs/use-docker-images-as-build-environments/ So no, we still need to create this folder before cdk command.

[github-actions[bot]]

⚠️COMMENT VISIBILITY WARNING⚠️

Comments on closed issues are hard for our team to see. If you need more assistance, please either tag a team member or open a new issue that references this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.