aws / aws-toolkit-vscode

Amazon Q, CodeCatalyst, Local Lambda debug, SAM/CFN syntax, ECS Terminal, AWS resources
https://marketplace.visualstudio.com/items?itemName=AmazonWebServices.amazon-q-vscode
Apache License 2.0
1.51k stars 439 forks source link

"Deploy SAM Application": Cannot build layer dependencies: "[Errno 30] Read-only file system" #2395

Closed miyuk closed 2 years ago

miyuk commented 2 years ago

Describe the bug

When use VS Command Deploy SAM Application with dependencies layer, build status is failed.

However, deploy process was continuing and create insufficient layer object(only requirements.txt file)

execution log is in additional context.

failed reason is [Errno 30] Read-only file system: '.aws-sam'.

when run command without AWS-ToolKit as below, dependencies ware resolved and create layer.

mkdir -p /tmp/aws-toolkit-vscode/samDeploytpFy2M/build 
 /usr/local/bin/sam build --build-dir /tmp/aws-toolkit-vscode/samDeploytpFy2M/build --template /Users/<USERNAME>/git/sam_test/lambda-python3.9/template.yaml

template.yml

Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.9
      Architectures:
        - x86_64
      Layers:
        - !Ref MyLayer
      Events:
        HelloWorld:
          Type: Api
          Properties:
            Path: /hello
            Method: get
  MyLayer:
    Type: AWS::Serverless::LayerVersion
    Properties:
      Description: Layer description
      ContentUri: my_layer/
      CompatibleRuntimes:
        - python3.9
    Metadata:
      BuildMethod: python3.9

my_layer/requirements.txt has only line: requests

To Reproduce

Steps to reproduce the behavior:

  1. create python SAM Application template with layer
  2. Add requirements.txt in layer directory
  3. run VS Command Deploy SAM Application

Expected behavior Build requirements.txt dependencies and create layer

Desktop (please complete the following information):

Additional context

sam_deploy.log

JadenSimon commented 2 years ago

Hey @miyuk,

Thanks for reporting this issue (and including a minimal example!). This looks like a problem with how SAM CLI caches layers. Do you have caching enabled? I only ask because I was unable to reproduce the issue until I turned on caching.

One workaround you can try is forcing the cache directory to be located in tmp by adding a samconfig.toml file to the root of the project:

version=0.1
[default.build.parameters]
cache_dir="/tmp/.aws-sam/cache"

The root-cause here is still unclear. Considering the command works when ran in a shell but fails when ran by the toolkit makes me think it's either a permissions issue or something with environment variables. If you could provides logs with your log level set to debug that would be very helpful! You can set the log level for the toolkit by going into VS Code's settings and searching for Aws: Log-Level.

miyuk commented 2 years ago

Thank you for reply.

I put samconfig.toml and change loglevel to DEBUG.

However, I couldn't reolve it. sam_deploy_2.log

JadenSimon commented 2 years ago

Looks like a permissions issue to me. Perhaps if you try to change the cache_dir to a folder in your workspace it'd work.

miyuk commented 2 years ago

I try change cache_dir to a folder in my workspace.

--cache_dir=/Users/<USERNAME>/git/sam_test/tmp/.aws-sam/cache 

samconfig.toml is in git/sam_test/lambda-python3.9 directory as below.

version=0.1
[default.build.parameters]
cache_dir="/Users/<USERNAME>/git/sam_test/tmp/.aws-sam/cache"

However build result is failed.

log ``` 2022-01-11 03:19:28,576 | --template_file=/Users//git/sam_test/lambda-python3.9/template.yaml --build_dir=/tmp/aws-toolkit-vscode/samDeployjw9wSF/build --cache_dir=/Users//git/sam_test/tmp/.aws-sam/cache 2022-01-11 03:19:28 [INFO]: 2022-01-11 03:19:28,985 | 'build' command is called 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,000 | No Parameters detected in the template 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,039 | 3 stacks found in the template 2022-01-11 03:19:29,039 | No Parameters detected in the template 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,064 | 3 resources found in the stack 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,064 | No Parameters detected in the template 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,087 | No Parameters detected in the template 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,112 | --base-dir is not presented, adjusting uri my_layer/ relative to /Users//git/sam_test/lambda-python3.9/template.yaml 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,113 | Found Serverless function with name='HelloWorldFunction' and CodeUri='hello_world/' 2022-01-11 03:19:29,113 | --base-dir is not presented, adjusting uri hello_world/ relative to /Users//git/sam_test/lambda-python3.9/template.yaml 2022-01-11 03:19:29,113 | No Parameters detected in the template 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,135 | --base-dir is not presented, adjusting uri my_layer/ relative to /Users//git/sam_test/lambda-python3.9/template.yaml 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,137 | Instantiating build definitions 2022-01-11 03:19:29,137 | No previous build graph found, generating new one 2022-01-11 03:19:29,137 | Unique function build definition found, adding as new (Function Build Definition: BuildDefinition(python3.9, /Users//git/sam_test/lambda-python3.9/hello_world, Zip, , cdd15a76-11a1-4d47-be57-bb982974e5d6, {}, {}, x86_64, []), Function: Function(name='HelloWorldFunction', functionname='HelloWorldFunction', runtime='python3.9', memory=None, timeout=3, handler='app.lambda_handler', imageuri=None, packagetype='Zip', imageconfig=None, codeuri='/Users//git/sam_test/lambda-python3.9/hello_world', environment=None, rolearn=None, layers=[], events={'HelloWorld': {'Type': 'Api', 'Properties': {'Path': '/hello', 'Method': 'get', 'RestApiId': 'ServerlessRestApi'}}}, metadata=None, inlinecode=None, codesign_config_arn=None, architectures=['x86_64'], stack_path='')) 2022-01-11 03:19:29,137 | Unique Layer build definition found, adding as new (Layer Build Definition: LayerBuildDefinition(MyLayer, /Users//git/sam_test/lambda-python3.9/my_layer, , 31ad2161-ada2-4c2c-8502-9de2c8d011aa, python3.9, ['python3.9'], x86_64, {}), Layer: ) 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,141 | Building codeuri: /Users//git/sam_test/lambda-python3.9/hello_world runtime: python3.9 metadata: {} architecture: x86_64 functions: ['HelloWorldFunction'] 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,144 | Building to following folder /private/tmp/aws-toolkit-vscode/samDeployjw9wSF/build/HelloWorldFunction 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,146 | Loading workflow module 'aws_lambda_builders.workflows' 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,157 | Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)' 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,161 | Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)' 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,164 | Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)' 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,169 | Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)' 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,175 | Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)' 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,183 | Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)' 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,188 | Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)' 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,197 | Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)' 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,202 | Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)' 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,202 | Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)' 2022-01-11 03:19:29,203 | requirements.txt file not found. Continuing the build without dependencies. 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,306 | Running workflow 'PythonPipBuilder' 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,307 | Running PythonPipBuilder:CopySource 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,315 | PythonPipBuilder:CopySource succeeded 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,315 | Building layer 'MyLayer' 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,317 | Loading workflow module 'aws_lambda_builders.workflows' 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,317 | Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)' 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,372 | Running workflow 'PythonPipBuilder' 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,373 | Running PythonPipBuilder:CleanUp 2022-01-11 03:19:29,373 | Clean up action: .aws-sam/deps/31ad2161-ada2-4c2c-8502-9de2c8d011aa does not exist and will be skipped. 2022-01-11 03:19:29,373 | PythonPipBuilder:CleanUp succeeded 2022-01-11 03:19:29,373 | Running PythonPipBuilder:ResolveDependencies 2022-01-11 03:19:29 [INFO]: 2022-01-11 03:19:29,425 | calling pip download -r /Users//git/sam_test/lambda-python3.9/my_layer/requirements.txt --dest /var/folders/0y/xl99vx814hnbc9xhqg8c43pm0000gn/T/tmpn499aesc --exists-action i 2022-01-11 03:19:31 [INFO]: 2022-01-11 03:19:31,182 | Full dependency closure: {charset-normalizer==2.0.10(wheel), idna==3.3(wheel), requests==2.27.1(wheel), certifi==2021.10.8(wheel), urllib3==1.26.8(wheel)} 2022-01-11 03:19:31 [INFO]: 2022-01-11 03:19:31,183 | initial compatible: {charset-normalizer==2.0.10(wheel), idna==3.3(wheel), requests==2.27.1(wheel), certifi==2021.10.8(wheel), urllib3==1.26.8(wheel)} 2022-01-11 03:19:31,184 | initial incompatible: set() 2022-01-11 03:19:31,184 | Downloading missing wheels: set() 2022-01-11 03:19:31 [INFO]: 2022-01-11 03:19:31,184 | compatible wheels after second download pass: {charset-normalizer==2.0.10(wheel), idna==3.3(wheel), requests==2.27.1(wheel), certifi==2021.10.8(wheel), urllib3==1.26.8(wheel)} 2022-01-11 03:19:31,184 | Build missing wheels from sdists (C compiling True): set() 2022-01-11 03:19:31 [INFO]: 2022-01-11 03:19:31,184 | compatible after building wheels (no C compiling): {charset-normalizer==2.0.10(wheel), idna==3.3(wheel), requests==2.27.1(wheel), certifi==2021.10.8(wheel), urllib3==1.26.8(wheel)} 2022-01-11 03:19:31,185 | Build missing wheels from sdists (C compiling False): set() 2022-01-11 03:19:31 [INFO]: 2022-01-11 03:19:31,185 | compatible after building wheels (C compiling): {charset-normalizer==2.0.10(wheel), idna==3.3(wheel), requests==2.27.1(wheel), certifi==2021.10.8(wheel), urllib3==1.26.8(wheel)} 2022-01-11 03:19:31,185 | Final compatible: {charset-normalizer==2.0.10(wheel), requests==2.27.1(wheel), certifi==2021.10.8(wheel), urllib3==1.26.8(wheel), idna==3.3(wheel)} 2022-01-11 03:19:31,185 | Final incompatible: set() 2022-01-11 03:19:31,185 | Final missing wheels: set() 2022-01-11 03:19:31 [INFO]: 2022-01-11 03:19:31,185 | PythonPipBuilder:ResolveDependencies raised unhandled exception Traceback (most recent call last): File "/usr/local/Cellar/aws-sam-cli/1.36.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflow.py", line 291, in run action.execute() File "/usr/local/Cellar/aws-sam-cli/1.36.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 51, in execute package_builder.build_dependencies( File "/usr/local/Cellar/aws-sam-cli/1.36.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 157, in build_dependencies self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path) File "/usr/local/Cellar/aws-sam-cli/1.36.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 259, in build_site_packages self._install_wheels(scratch_directory, target_directory, wheels) File "/usr/local/Cellar/aws-sam-cli/1.36.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 502, in _install_wheels self._osutils.makedirs(dst_dir) File "/usr/local/Cellar/aws-sam-cli/1.36.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/utils.py", line 67, in makedirs os.makedirs(path) File "/usr/local/Cellar/python@3.8/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py", line 213, in makedirs makedirs(head, exist_ok=exist_ok) File "/usr/local/Cellar/python@3.8/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py", line 213, in makedirs makedirs(head, exist_ok=exist_ok) File "/usr/local/Cellar/python@3.8/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py", line 223, in makedirs mkdir(name, mode) OSError: [Errno 30] Read-only file system: '.aws-sam' 2022-01-11 03:19:31 [INFO]: Build Failed 2022-01-11 03:19:31 [INFO]: 2022-01-11 03:19:31,194 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': '11d71b9a-b7f6-4cb7-bfd7-19d394560b32', 'installationId': 'be4bbf94-8b54-4a7e-87e4-f094de2f2802', 'sessionId': '3f3e27d1-9417-405d-ba45-d146b2fbaab1', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.8.12', 'samcliVersion': '1.36.0', 'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'duration': 2618, 'exitReason': 'WorkflowUnknownError', 'exitCode': 1}}]} 2022-01-11 03:19:31 [INFO]: 2022-01-11 03:19:31,712 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1) 2022-01-11 03:19:31 [INFO]: Error: PythonPipBuilder:ResolveDependencies - [Errno 30] Read-only file system: '.aws-sam' 2022-01-11 03:19:31 [WARN]: "sam build" failed: /Users//git/sam_test/lambda-python3.9/template.yaml 2022-01-11 03:19:31 [INFO]: Packaging SAM Application to S3 Bucket: helloworld-sam-test ```
JadenSimon commented 2 years ago

Looks like it's failing on downloading dependencies to an auto-generated build directory now:

2022-01-04 23:21:57 [INFO]: 2022-01-04 23:21:57,653 | Building to following folder /private/tmp/aws-toolkit-vscode/samDeploy1l3tWg/build/HelloWorldFunction

We currently have no way to configure this apart of a deploy workflow unfortunately. Can you check what permissions your /private/tmp directory has?

miyuk commented 2 years ago

sorry for late reply

permission of my /private/tmp is below user is root and all flag permission

MAC-2:~ xxxxx$ ls -la /private/tmp/
total 0
drwxrwxrwt  12 root   wheel  384  2  6 09:10 .
drwxr-xr-x   6 root   wheel  192  1  4 23:39 ..
drwx------   3 xxxxx  wheel   96  1 11 03:07 com.apple.launchd.09sNOsS7Yu
drwx------   3 xxxxx  wheel   96  1 30 20:33 com.apple.launchd.XpBEgNVjDg
drwx------   3 xxxxx  wheel   96  1 30 20:33 com.apple.launchd.eNVUvWSJGB
drwx------   3 xxxxx  wheel   96  1 11 03:07 com.apple.launchd.kBLD8lyD5D
drwx------   3 xxxxx  wheel   96  1 12 19:57 com.apple.launchd.p1pAsfWVQs
drwx------   3 xxxxx  wheel   96  1 12 19:57 com.apple.launchd.tekm0ykNsY
drwxr-xr-x@  4 xxxxx  wheel  128  1 30 20:33 com.google.Keystone
srwxrwxrwx   1 xxxxx  wheel    0  1 30 20:33 fctvpnctl.sock
-rw-------   1 root   wheel    0  2  6 09:10 periodic.UKzdrxZbc9
drwxr-xr-x   2 root   wheel   64  2  6 09:09 powerlog
plantrackretire commented 2 years ago

Hi, I'm getting the exact same error using VS Code. Currently just building a simple layer and function to test it. If I comment out the layer in the template and set the layer to the ARN of a version already installed on AWS then it builds and runs, it only fails when including the layer in the build. My output is below, happy to provide any additional info. I also checked /private/tmp as mentioned above and I see output there from the build:

build log peters-mbp-2:code peter$ ls -lrt /private/tmp/aws-toolkit-vscode/vsctkx3egkJ/output/CommonLayerTestFunction total 8 -rw-r--r-- 1 peter wheel 824 Mar 5 09:06 lambda_function.py drwxr-xr-x 2 peter wheel 64 Mar 5 10:05 debugpy.libs drwxr-xr-x 11 peter wheel 352 Mar 5 10:05 debugpy drwxr-xr-x 7 peter wheel 224 Mar 5 10:05 debugpy-1.5.1.dist-info And I checked /var/folders/pd/49379p0d52v708ryfhrz24j00000gn/T/tmpzyqn5dgv, which is mentioned in the output, that folder does not exist. The 'T' folder exists: peters-mbp-2:code peter$ ls -lrt /var/folders/pd/49379p0d52v708ryfhrz24j00000gn total 0 drwxr-xr-x 2 peter staff 64 Oct 5 2017 Cleanup At Startup drwxr-xr-x@ 28 peter staff 896 Apr 28 2021 0 drwxr-xr-x 2 peter staff 64 Jul 23 2021 X drwx------@ 174 peter staff 5568 Mar 5 08:53 C drwx------@ 134 peter staff 4288 Mar 5 17:00 T And here is the output from VS Code: 2022-03-05 10:05:50 [INFO]: Preparing to debug locally: Lambda "" 2022-03-05 10:05:50 [INFO]: Building SAM application... 2022-03-05 10:05:51 [INFO]: Running command: (not started) [/usr/local/bin/sam build --debug --build-dir /tmp/aws-toolkit-vscode/vsctkx3egkJ/output --template /Users/peter/Documents/code/test-lambda/template.yaml --manifest /tmp/aws-toolkit-vscode/vsctkx3egkJ/debug-requirements.txt] 2022-03-05 10:05:51 [INFO]: 2022-03-05 10:05:51,982 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics 2022-03-05 10:05:51,982 | Using config file: samconfig.toml, config environment: default 2022-03-05 10:05:51 [INFO]: 2022-03-05 10:05:51,983 | Expand command line arguments to: 2022-03-05 10:05:51,983 | --template_file=/Users/peter/Documents/code/test-lambda/template.yaml --build_dir=/tmp/aws-toolkit-vscode/vsctkx3egkJ/output --manifest=/tmp/aws-toolkit-vscode/vsctkx3egkJ/debug-requirements.txt --cache_dir=.aws-sam/cache 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,591 | 'build' command is called 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,603 | Collected default values for parameters: {'LambdaRole': 'arn:aws:iam::XXXXXXXXXX:role/lambda-etl-role'} 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,623 | There is no customer defined id or cdk path defined for resource CommonLayer, so we will use the resource logical id as the resource id 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,623 | There is no customer defined id or cdk path defined for resource CommonLayerTestFunction, so we will use the resource logical id as the resource id 2022-03-05 10:05:52,624 | 2 stacks found in the template 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,624 | Collected default values for parameters: {'LambdaRole': 'arn:aws:iam::XXXXXXXXXX:role/lambda-etl-role'} 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,638 | There is no customer defined id or cdk path defined for resource CommonLayer, so we will use the resource logical id as the resource id 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,638 | There is no customer defined id or cdk path defined for resource CommonLayerTestFunction, so we will use the resource logical id as the resource id 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,639 | 2 resources found in the stack 2022-03-05 10:05:52,639 | Collected default values for parameters: {'LambdaRole': 'arn:aws:iam::XXXXXXXXXX:role/lambda-etl-role'} 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,652 | There is no customer defined id or cdk path defined for resource CommonLayer, so we will use the resource logical id as the resource id 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,652 | There is no customer defined id or cdk path defined for resource CommonLayerTestFunction, so we will use the resource logical id as the resource id 2022-03-05 10:05:52,653 | Found Serverless function with name='CommonLayerTestFunction' and CodeUri='functions/layer-test--common/src' 2022-03-05 10:05:52,653 | --base-dir is not presented, adjusting uri functions/layer-test--common/src relative to /Users/peter/Documents/code/test-lambda/template.yaml 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,653 | Collected default values for parameters: {'LambdaRole': 'arn:aws:iam::XXXXXXXXXX:role/lambda-etl-role'} 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,666 | There is no customer defined id or cdk path defined for resource CommonLayer, so we will use the resource logical id as the resource id 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,666 | There is no customer defined id or cdk path defined for resource CommonLayerTestFunction, so we will use the resource logical id as the resource id 2022-03-05 10:05:52,666 | --base-dir is not presented, adjusting uri layers/common/src relative to /Users/peter/Documents/code/test-lambda/template.yaml 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,668 | Instantiating build definitions 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,668 | No previous build graph found, generating new one 2022-03-05 10:05:52,668 | Unique function build definition found, adding as new (Function Build Definition: BuildDefinition(python3.9, /Users/peter/Documents/code/test-lambda/functions/layer-test--common/src, Zip, , 22dacffe-582c-47a3-9ddc-244ac3bed013, {}, {}, x86_64, []), Function: Function(function_id='CommonLayerTestFunction', name='CommonLayerTestFunction', functionname='test-layer-test--common', runtime='python3.9', memory=128, timeout=180, handler='lambda_function.lambda_handler', imageuri=None, packagetype='Zip', imageconfig=None, codeuri='/Users/peter/Documents/code/test-lambda/functions/layer-test--common/src', environment={'Variables': {'DB_PATH': 'DbPath'}}, rolearn='arn:aws:iam::XXXXXXXXXX:role/lambda-etl-role', layers=[], events=None, metadata={'SamResourceId': 'CommonLayerTestFunction'}, inlinecode=None, codesign_config_arn=None, architectures=['x86_64'], stack_path='')) 2022-03-05 10:05:52,668 | Unique Layer build definition found, adding as new (Layer Build Definition: LayerBuildDefinition(CommonLayer, /Users/peter/Documents/code/test-lambda/layers/common/src, , bdcf0318-f649-4225-b571-1360385cfcee, python3.9, ['python3.9'], x86_64, {}), Layer: ) 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,670 | Building codeuri: /Users/peter/Documents/code/test-lambda/functions/layer-test--common/src runtime: python3.9 metadata: {} architecture: x86_64 functions: ['CommonLayerTestFunction'] 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,670 | Building to following folder /private/tmp/aws-toolkit-vscode/vsctkx3egkJ/output/CommonLayerTestFunction 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,672 | Loading workflow module 'aws_lambda_builders.workflows' 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,677 | Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)' 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,680 | Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)' 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,683 | Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)' 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,687 | Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)' 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,692 | Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)' 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,697 | Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)' 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,700 | Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)' 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,704 | Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)' 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,708 | Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)' 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,708 | Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)' 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,737 | Running workflow 'PythonPipBuilder' 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,738 | Running PythonPipBuilder:ResolveDependencies 2022-03-05 10:05:52 [INFO]: 2022-03-05 10:05:52,801 | calling pip download -r /tmp/aws-toolkit-vscode/vsctkx3egkJ/debug-requirements.txt --dest /var/folders/pd/49379p0d52v708ryfhrz24j00000gn/T/tmpzyqn5dgv --exists-action i 2022-03-05 10:05:53 [INFO]: 2022-03-05 10:05:53,971 | Full dependency closure: {debugpy==1.5.1(wheel)} 2022-03-05 10:05:53 [INFO]: 2022-03-05 10:05:53,972 | initial compatible: set() 2022-03-05 10:05:53,973 | initial incompatible: {debugpy==1.5.1(wheel)} 2022-03-05 10:05:53,973 | Downloading missing wheels: {debugpy==1.5.1(wheel)} 2022-03-05 10:05:53 [INFO]: 2022-03-05 10:05:53,973 | calling pip download --only-binary=:all: --no-deps --platform manylinux2014_x86_64 --implementation cp --abi cp39 --dest /var/folders/pd/49379p0d52v708ryfhrz24j00000gn/T/tmpzyqn5dgv debugpy==1.5.1 2022-03-05 10:05:55 [INFO]: 2022-03-05 10:05:55,067 | compatible wheels after second download pass: {debugpy==1.5.1(wheel)} 2022-03-05 10:05:55,068 | Build missing wheels from sdists (C compiling True): set() 2022-03-05 10:05:55 [INFO]: 2022-03-05 10:05:55,069 | compatible after building wheels (no C compiling): {debugpy==1.5.1(wheel)} 2022-03-05 10:05:55 [INFO]: 2022-03-05 10:05:55,069 | Build missing wheels from sdists (C compiling False): set() 2022-03-05 10:05:55,070 | compatible after building wheels (C compiling): {debugpy==1.5.1(wheel)} 2022-03-05 10:05:55,070 | Final compatible: {debugpy==1.5.1(wheel)} 2022-03-05 10:05:55 [INFO]: 2022-03-05 10:05:55,070 | Final incompatible: {debugpy==1.5.1(wheel)} 2022-03-05 10:05:55,070 | Final missing wheels: set() 2022-03-05 10:05:55 [INFO]: 2022-03-05 10:05:55,395 | PythonPipBuilder:ResolveDependencies succeeded 2022-03-05 10:05:55 [INFO]: 2022-03-05 10:05:55,395 | Running PythonPipBuilder:CopySource 2022-03-05 10:05:55 [INFO]: 2022-03-05 10:05:55,404 | PythonPipBuilder:CopySource succeeded 2022-03-05 10:05:55 [INFO]: 2022-03-05 10:05:55,409 | Building layer 'CommonLayer' 2022-03-05 10:05:55 [INFO]: 2022-03-05 10:05:55,410 | Loading workflow module 'aws_lambda_builders.workflows' 2022-03-05 10:05:55 [INFO]: 2022-03-05 10:05:55,411 | Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)' 2022-03-05 10:05:55 [INFO]: 2022-03-05 10:05:55,453 | Running workflow 'PythonPipBuilder' 2022-03-05 10:05:55 [INFO]: 2022-03-05 10:05:55,454 | Running PythonPipBuilder:CleanUp 2022-03-05 10:05:55,454 | Clean up action: .aws-sam/deps/bdcf0318-f649-4225-b571-1360385cfcee does not exist and will be skipped. 2022-03-05 10:05:55,455 | PythonPipBuilder:CleanUp succeeded 2022-03-05 10:05:55,455 | Running PythonPipBuilder:ResolveDependencies 2022-03-05 10:05:55 [INFO]: 2022-03-05 10:05:55,551 | calling pip download -r /tmp/aws-toolkit-vscode/vsctkx3egkJ/debug-requirements.txt --dest /var/folders/pd/49379p0d52v708ryfhrz24j00000gn/T/tmply8be58o --exists-action i 2022-03-05 10:05:57 [INFO]: 2022-03-05 10:05:57,035 | Full dependency closure: {debugpy==1.5.1(wheel)} 2022-03-05 10:05:57 [INFO]: 2022-03-05 10:05:57,035 | initial compatible: set() 2022-03-05 10:05:57,036 | initial incompatible: {debugpy==1.5.1(wheel)} 2022-03-05 10:05:57,036 | Downloading missing wheels: {debugpy==1.5.1(wheel)} 2022-03-05 10:05:57,036 | calling pip download --only-binary=:all: --no-deps --platform manylinux2014_x86_64 --implementation cp --abi cp39 --dest /var/folders/pd/49379p0d52v708ryfhrz24j00000gn/T/tmply8be58o debugpy==1.5.1 2022-03-05 10:05:58 [INFO]: 2022-03-05 10:05:58,746 | compatible wheels after second download pass: {debugpy==1.5.1(wheel)} 2022-03-05 10:05:58 [INFO]: 2022-03-05 10:05:58,747 | Build missing wheels from sdists (C compiling True): set() 2022-03-05 10:05:58,747 | compatible after building wheels (no C compiling): {debugpy==1.5.1(wheel)} 2022-03-05 10:05:58,747 | Build missing wheels from sdists (C compiling False): set() 2022-03-05 10:05:58 [INFO]: 2022-03-05 10:05:58,748 | compatible after building wheels (C compiling): {debugpy==1.5.1(wheel)} 2022-03-05 10:05:58,748 | Final compatible: {debugpy==1.5.1(wheel)} 2022-03-05 10:05:58,748 | Final incompatible: {debugpy==1.5.1(wheel)} 2022-03-05 10:05:58,748 | Final missing wheels: set() 2022-03-05 10:05:58 [INFO]: 2022-03-05 10:05:58,748 | PythonPipBuilder:ResolveDependencies raised unhandled exception Traceback (most recent call last): File "/usr/local/Cellar/aws-sam-cli/1.37.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflow.py", line 291, in run action.execute() File "/usr/local/Cellar/aws-sam-cli/1.37.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/actions.py", line 51, in execute package_builder.build_dependencies( File "/usr/local/Cellar/aws-sam-cli/1.37.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 157, in build_dependencies self._dependency_builder.build_site_packages(requirements_path, artifacts_dir_path, scratch_dir_path) File "/usr/local/Cellar/aws-sam-cli/1.37.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 259, in build_site_packages self._install_wheels(scratch_directory, target_directory, wheels) File "/usr/local/Cellar/aws-sam-cli/1.37.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/packager.py", line 502, in _install_wheels self._osutils.makedirs(dst_dir) File "/usr/local/Cellar/aws-sam-cli/1.37.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflows/python_pip/utils.py", line 67, in makedirs os.makedirs(path) File "/usr/local/Cellar/python@3.8/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py", line 213, in makedirs makedirs(head, exist_ok=exist_ok) File "/usr/local/Cellar/python@3.8/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py", line 213, in makedirs makedirs(head, exist_ok=exist_ok) File "/usr/local/Cellar/python@3.8/3.8.12_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py", line 223, in makedirs mkdir(name, mode) OSError: [Errno 30] Read-only file system: '.aws-sam' 2022-03-05 10:05:58 [INFO]: Build Failed 2022-03-05 10:05:58 [INFO]: 2022-03-05 10:05:58,772 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': '147411a4-7e72-41c9-85c1-c836eadb69a4', 'installationId': 'bebd84da-0864-4daa-aeca-c66c73e52aa3', 'sessionId': 'bb4d4028-eeb2-4c81-9f54-f45d295d296a', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.8.12', 'samcliVersion': '1.37.0', 'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'metricSpecificAttributes': {'projectType': 'CFN'}, 'duration': 6789, 'exitReason': 'WorkflowUnknownError', 'exitCode': 1}}]} 2022-03-05 10:05:59 [INFO]: 2022-03-05 10:05:59,162 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1) 2022-03-05 10:05:59 [INFO]: Error: PythonPipBuilder:ResolveDependencies - [Errno 30] Read-only file system: '.aws-sam' 2022-03-05 10:05:59 [WARN]: "sam build" failed: /Users/peter/Documents/code/test-lambda/template.yaml
justinmk3 commented 2 years ago

OSError: [Errno 30] Read-only file system: '.aws-sam'

@plantrackretire this looks like the filesystem or disk might be corrupted. Or user doesn't have write access to the filesystem (would be unusual).

Is this behavior not occurring with any other programs on your computer?

plantrackretire commented 2 years ago

thanks for taking the time to respond @justinmk3

I have not been having any other issues, though I don't do much coding beyond this area. I am able to build, invoke, and deploy the template from the command line with sam directly. And it works if there is no layer involved. Is there a unique directory involved when a layer is in the template? Maybe I have an issue with permissions one that folder? Not clear to me as the debug logging isn't saying exactly where the read only issue is happening. Appreciate any help and suggestions people can provide, I'm stuck one this one. My other option is to split the layers into their own template, build and deploy them on the command line, hard code the arn for the layers in the lambda functions template and only use vs code for those. But that would be unfortunate as I'm trying to avoid hardcoding the layer and version and I lose the ability to debug the layers.

Thought my error was pretty much the same as the original poster, but maybe I missed a key difference.

Thanks again

Fabian-JP commented 2 years ago

I'm also hitting this problem since I have formatted my mac. It used to work, not after reinstalling macOS it does not work. It seems adding a samconfig.toml to my root directory or my src directory does nothing to change the cache_dir. Is there something else I need to do to get it to detect the samconfig.toml?

Snippet from my logs:

`2022-07-07 14:37:12 [INFO]: Building SAM application... 2022-07-07 14:37:12 [INFO]: Command: (not started) [/usr/local/bin/sam build --debug --build-dir /tmp/aws-toolkit-vscode/vsctkB5jyaT/output --template /tmp/aws-toolkit-vscode/vsctkB5jyaT/appvsctktemplate.yaml --base-dir /Users/fabian/Projects/sanfl-backend-api/src] 2022-07-07 14:37:13 [INFO]: 2022-07-07 14:37:13,422 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics

2022-07-07 14:37:13 [INFO]: 2022-07-07 14:37:13,422 | Using config file: samconfig.toml, config environment: default 2022-07-07 14:37:13,422 | Expand command line arguments to: 2022-07-07 14:37:13,422 | --template_file=/tmp/aws-toolkit-vscode/vsctkB5jyaT/appvsctktemplate.yaml --build_dir=/tmp/aws-toolkit-vscode/vsctkB5jyaT/output --base_dir=/Users/fabian/Projects/sanfl-backend-api/src --cache_dir=.aws-sam/cache `

Additionally I've tried adding this to my launch.json configurations:

"sam": { "buildArguments": ["--cache-dir=/Users/fabian/aws-temp/cache"] }

which seems to correctly update the argument supplied to sam build, but .aws-sam directory is still trying to be accessed for some reason and failing:

`2022-07-07 14:54:44 [INFO]: Building SAM application... 2022-07-07 14:54:44 [INFO]: Command: (not started) [/usr/local/bin/sam build --debug --build-dir /tmp/aws-toolkit-vscode/vsctkoZCTH6/output --template /tmp/aws-toolkit-vscode/vsctkoZCTH6/appvsctktemplate.yaml --base-dir /Users/fabian/Projects/sanfl-backend-api/src --cache-dir=/Users/fabian/aws-temp/cache] 2022-07-07 14:54:44 [INFO]: 2022-07-07 14:54:44,916 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics

2022-07-07 14:54:44 [INFO]: 2022-07-07 14:54:44,916 | Using config file: samconfig.toml, config environment: default 2022-07-07 14:54:44,917 | Expand command line arguments to: 2022-07-07 14:54:44,917 | --template_file=/tmp/aws-toolkit-vscode/vsctkoZCTH6/appvsctktemplate.yaml --build_dir=/tmp/aws-toolkit-vscode/vsctkoZCTH6/output --base_dir=/Users/fabian/Projects/sanfl-backend-api/src --cache_dir=/Users/fabian/aws-temp/cache

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,345 | 'build' command is called

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,348 | No Parameters detected in the template

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,370 | There is no customer defined id or cdk path defined for resource src, so we will use the resource logical id as the resource id

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,370 | 0 stacks found in the template 2022-07-07 14:54:45,371 | No Parameters detected in the template

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,386 | There is no customer defined id or cdk path defined for resource src, so we will use the resource logical id as the resource id

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,386 | 1 resources found in the stack 2022-07-07 14:54:45,386 | Found Serverless function with name='src' and CodeUri='/Users/fabian/Projects/sanfl-backend-api/src'

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,389 | Instantiating build definitions

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,389 | No previous build graph found, generating new one 2022-07-07 14:54:45,390 | Unique function build definition found, adding as new (Function Build Definition: BuildDefinition(nodejs14.x, /Users/fabian/Projects/sanfl-backend-api/src, Zip, , 01415133-f893-4f66-9381-250609ddfc24, {}, {}, x86_64, []), Function: Function(function_id='src', name='src', functionname='src', runtime='nodejs14.x', memory=None, timeout=None, handler='functionTests/test_reprocessCurrentSeasonPlayerStats.test_reprocessCurrentSeasonPlayerStats', imageuri=None, packagetype='Zip', imageconfig=None, codeuri='/Users/fabian/Projects/sanfl-backend-api/src', environment={'Variables': OrderedDict()}, rolearn=None, layers=[], events=None, metadata={'SamResourceId': 'src'}, inlinecode=None, codesign_config_arn=None, architectures=None, function_url_config=None, stack_path=''))

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,391 | Building codeuri: /Users/fabian/Projects/sanfl-backend-api/src runtime: nodejs14.x metadata: {} architecture: x86_64 functions: src

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,391 | Building to following folder /private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,397 | Loading workflow module 'aws_lambda_builders.workflows'

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,400 | Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,402 | Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,405 | Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,407 | Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,410 | Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,412 | Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,414 | Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,416 | Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,418 | Registering workflow 'NodejsNpmEsbuildBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm-esbuild', application_framework=None)'

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,418 | Found workflow 'NodejsNpmBuilder' to support capabilities 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,419 | Running workflow 'NodejsNpmBuilder'

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,419 | Running NodejsNpmBuilder:NpmPack 2022-07-07 14:54:45,419 | NODEJS packaging file:/Users/fabian/Projects/sanfl-backend-api/src to /var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk 2022-07-07 14:54:45,419 | executing NPM: ['npm', 'pack', '-q', 'file:/Users/fabian/Projects/sanfl-backend-api/src']

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,980 | NODEJS packed to common-libraries-0.0.1.tgz

2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,982 | NODEJS extracting to /var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked

2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,016 | NodejsNpmBuilder:NpmPack succeeded

2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,017 | Running NodejsNpmBuilder:CopyNpmrcAndLockfile 2022-07-07 14:54:46,017 | package-lock.json copying in: /var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked/package

2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,018 | NodejsNpmBuilder:CopyNpmrcAndLockfile succeeded

2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,018 | Running NodejsNpmBuilder:CopySource 2022-07-07 14:54:46,018 | Creating target folders at /private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src

2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,018 | Copying directory metadata from source (/var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked/package) to destination (/private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src)

2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,019 | Creating target folders at /private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src/tests

2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,019 | Copying directory metadata from source (/var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked/package/tests) to destination (/private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src/tests)

2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,019 | Copying source file (/var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked/package/tests/test_appNews.js) to destination (/private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src/tests/test_appNews.js)

2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,020 | Copying source file (/var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked/package/tests/test_clubAds.js) to destination (/private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src/tests/test_clubAds.js)

...

2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,066 | Copying source file (/var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked/package/data/fixtureMethods.js) to destination (/private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src/data/fixtureMethods.js)

2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,067 | Copying source file (/var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked/package/data/playerMethods.js) to destination (/private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src/data/playerMethods.js)

2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,067 | NodejsNpmBuilder:CopySource succeeded

2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,068 | Running NodejsNpmBuilder:NpmInstall 2022-07-07 14:54:46,068 | NODEJS installing in: /private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src 2022-07-07 14:54:46,068 | executing NPM: ['npm', 'install', '-q', '--no-audit', '--no-save', '--production', '--unsafe-perm']

2022-07-07 14:54:47 [INFO]: 2022-07-07 14:54:47,618 | NodejsNpmBuilder:NpmInstall succeeded

2022-07-07 14:54:47 [INFO]: 2022-07-07 14:54:47,619 | Running NodejsNpmBuilder:CleanUp 2022-07-07 14:54:47,619 | Clean up action: .aws-sam/deps/01415133-f893-4f66-9381-250609ddfc24 does not exist and will be skipped. 2022-07-07 14:54:47,619 | NodejsNpmBuilder:CleanUp succeeded 2022-07-07 14:54:47,619 | Running NodejsNpmBuilder:CopyDependencies

2022-07-07 14:54:47 [INFO]: 2022-07-07 14:54:47,620 | Creating target folders at .aws-sam/deps/01415133-f893-4f66-9381-250609ddfc24/node_modules

2022-07-07 14:54:47 [INFO]: 2022-07-07 14:54:47,620 | NodejsNpmBuilder:CopyDependencies raised unhandled exception Traceback (most recent call last): File "/usr/local/Cellar/aws-sam-cli/1.53.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflow.py", line 301, in run action.execute() File "/usr/local/Cellar/aws-sam-cli/1.53.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/actions.py", line 132, in execute copytree(dependencies_source, new_destination) File "/usr/local/Cellar/aws-sam-cli/1.53.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/utils.py", line 46, in copytree os.makedirs(destination) File "/usr/local/Cellar/python@3.8/3.8.13_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py", line 213, in makedirs makedirs(head, exist_ok=exist_ok) File "/usr/local/Cellar/python@3.8/3.8.13_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py", line 213, in makedirs makedirs(head, exist_ok=exist_ok) File "/usr/local/Cellar/python@3.8/3.8.13_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py", line 213, in makedirs makedirs(head, exist_ok=exist_ok) File "/usr/local/Cellar/python@3.8/3.8.13_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py", line 223, in makedirs mkdir(name, mode) OSError: [Errno 30] Read-only file system: '.aws-sam'

2022-07-07 14:54:47 [INFO]: Build Failed

2022-07-07 14:54:47 [INFO]: 2022-07-07 14:54:47,633 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': 'e8325380-45a9-4768-bd3b-9d8fd9bef8de', 'installationId': 'da55d10c-476a-4985-893e-5e6fa7142ad5', 'sessionId': 'ebfb26b0-5f89-46fb-917e-d633a6c7b6fc', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.8.13', 'samcliVersion': '1.53.0', 'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'metricSpecificAttributes': {'projectType': 'CFN'}, 'duration': 2716, 'exitReason': 'WorkflowUnknownError', 'exitCode': 1}}]}

2022-07-07 14:54:49 [INFO]: 2022-07-07 14:54:49,411 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)

2022-07-07 14:54:49 [INFO]: Error: NodejsNpmBuilder:CopyDependencies - [Errno 30] Read-only file system: '.aws-sam'

2022-07-07 14:54:49 [WARN]: "sam build" failed: /tmp/aws-toolkit-vscode/vsctkoZCTH6/appvsctktemplate.yaml`

JadenSimon commented 2 years ago

Currently, I think these problems are because of SAM CLI's reliance on the current working directory for significant portions of its config resolution logic. The Toolkit does not provide an explicit working directory which can lead to all sorts of weird issues, especially when combined with the generated temporary directory (see https://github.com/aws/aws-toolkit-vscode/issues/2050).

The solution could be as simple as setting the current working directory to the SAM project, though this may have unintended consequences. Work is in progress for some sort of fix.

ryancabanas commented 2 years ago

@JadenSimon

For my issue (that I wrongly posted here) that's somewhat related to this one, I just wanted to let you know that rolling back to aws-sam-cli 1.52.0 fixed the issue for me. Thanks!

JadenSimon commented 2 years ago

https://github.com/aws/aws-toolkit-vscode/pull/2755 is a potential solution though it would be very helpful if anyone experiencing this problem could try out the fix:

  1. Download and extract the following VSIX: aws-toolkit-vscode-1.46.0-3e2137d9f381.vsix.zip
  2. In VS Code, run the Extensions: Install from VSIX... command to install the VSIX.
    • The default shortcut for opening the command palette is ctrl-shift-p for Windows/Linux and cmd-shift-p for Mac
  3. Reload VS Code and verify that the Toolkit has the new version.
awied commented 2 years ago

Installing the above VSIX fixes the issue for me while using the latest version of AWS SAM CLI (v1.53.0). Thank you!

ryancabanas commented 2 years ago

@JadenSimon

I too confirm that the above VSIX allows me to, once again, successfully use debugging with aws-sam-cli v1.53.0. Thanks!

I take it this AWS Toolkit preview will simply be overwritten once the official 1.46.0 version is released? Thanks!

JadenSimon commented 2 years ago

I take it this AWS Toolkit preview will simply be overwritten once the official 1.46.0 version is released? Thanks!

Yeah it should, at least from my experience. If it doesn't do it automatically then you can manually upgrade using the extensions tab.

Fabian-JP commented 2 years ago

I also can confirm this VSIX works for me. It did initially complain the ~/.aws-sam directory was read only so I deleted it manually then it worked.

adam-weiss-tfs commented 2 years ago

This VSIX worked for me as well with aws-sam-cli 1.53.0. Thanks!

justinmk3 commented 2 years ago

AWS Toolkit 1.46 includes a fix for this.