Closed miyuk closed 2 years ago
Hey @miyuk,
Thanks for reporting this issue (and including a minimal example!). This looks like a problem with how SAM CLI caches layers. Do you have caching enabled? I only ask because I was unable to reproduce the issue until I turned on caching.
One workaround you can try is forcing the cache directory to be located in tmp
by adding a samconfig.toml
file to the root of the project:
version=0.1
[default.build.parameters]
cache_dir="/tmp/.aws-sam/cache"
The root-cause here is still unclear. Considering the command works when ran in a shell but fails when ran by the toolkit makes me think it's either a permissions issue or something with environment variables. If you could provides logs with your log level set to debug
that would be very helpful! You can set the log level for the toolkit by going into VS Code's settings and searching for Aws: Log-Level
.
Thank you for reply.
I put samconfig.toml
and change loglevel to DEBUG
.
However, I couldn't reolve it. sam_deploy_2.log
Looks like a permissions issue to me. Perhaps if you try to change the cache_dir
to a folder in your workspace it'd work.
I try change cache_dir
to a folder in my workspace.
--cache_dir=/Users/<USERNAME>/git/sam_test/tmp/.aws-sam/cache
samconfig.toml is in git/sam_test/lambda-python3.9
directory as below.
version=0.1
[default.build.parameters]
cache_dir="/Users/<USERNAME>/git/sam_test/tmp/.aws-sam/cache"
However build result is failed.
Looks like it's failing on downloading dependencies to an auto-generated build directory now:
2022-01-04 23:21:57 [INFO]: 2022-01-04 23:21:57,653 | Building to following folder /private/tmp/aws-toolkit-vscode/samDeploy1l3tWg/build/HelloWorldFunction
We currently have no way to configure this apart of a deploy workflow unfortunately. Can you check what permissions your /private/tmp
directory has?
sorry for late reply
permission of my /private/tmp
is below
user is root and all flag permission
MAC-2:~ xxxxx$ ls -la /private/tmp/
total 0
drwxrwxrwt 12 root wheel 384 2 6 09:10 .
drwxr-xr-x 6 root wheel 192 1 4 23:39 ..
drwx------ 3 xxxxx wheel 96 1 11 03:07 com.apple.launchd.09sNOsS7Yu
drwx------ 3 xxxxx wheel 96 1 30 20:33 com.apple.launchd.XpBEgNVjDg
drwx------ 3 xxxxx wheel 96 1 30 20:33 com.apple.launchd.eNVUvWSJGB
drwx------ 3 xxxxx wheel 96 1 11 03:07 com.apple.launchd.kBLD8lyD5D
drwx------ 3 xxxxx wheel 96 1 12 19:57 com.apple.launchd.p1pAsfWVQs
drwx------ 3 xxxxx wheel 96 1 12 19:57 com.apple.launchd.tekm0ykNsY
drwxr-xr-x@ 4 xxxxx wheel 128 1 30 20:33 com.google.Keystone
srwxrwxrwx 1 xxxxx wheel 0 1 30 20:33 fctvpnctl.sock
-rw------- 1 root wheel 0 2 6 09:10 periodic.UKzdrxZbc9
drwxr-xr-x 2 root wheel 64 2 6 09:09 powerlog
Hi, I'm getting the exact same error using VS Code. Currently just building a simple layer and function to test it. If I comment out the layer in the template and set the layer to the ARN of a version already installed on AWS then it builds and runs, it only fails when including the layer in the build. My output is below, happy to provide any additional info. I also checked /private/tmp as mentioned above and I see output there from the build:
OSError: [Errno 30] Read-only file system: '.aws-sam'
@plantrackretire this looks like the filesystem or disk might be corrupted. Or user doesn't have write access to the filesystem (would be unusual).
Is this behavior not occurring with any other programs on your computer?
thanks for taking the time to respond @justinmk3
I have not been having any other issues, though I don't do much coding beyond this area. I am able to build, invoke, and deploy the template from the command line with sam directly. And it works if there is no layer involved. Is there a unique directory involved when a layer is in the template? Maybe I have an issue with permissions one that folder? Not clear to me as the debug logging isn't saying exactly where the read only issue is happening. Appreciate any help and suggestions people can provide, I'm stuck one this one. My other option is to split the layers into their own template, build and deploy them on the command line, hard code the arn for the layers in the lambda functions template and only use vs code for those. But that would be unfortunate as I'm trying to avoid hardcoding the layer and version and I lose the ability to debug the layers.
Thought my error was pretty much the same as the original poster, but maybe I missed a key difference.
Thanks again
I'm also hitting this problem since I have formatted my mac. It used to work, not after reinstalling macOS it does not work. It seems adding a samconfig.toml to my root directory or my src directory does nothing to change the cache_dir. Is there something else I need to do to get it to detect the samconfig.toml?
Snippet from my logs:
`2022-07-07 14:37:12 [INFO]: Building SAM application... 2022-07-07 14:37:12 [INFO]: Command: (not started) [/usr/local/bin/sam build --debug --build-dir /tmp/aws-toolkit-vscode/vsctkB5jyaT/output --template /tmp/aws-toolkit-vscode/vsctkB5jyaT/appvsctktemplate.yaml --base-dir /Users/fabian/Projects/sanfl-backend-api/src] 2022-07-07 14:37:13 [INFO]: 2022-07-07 14:37:13,422 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2022-07-07 14:37:13 [INFO]: 2022-07-07 14:37:13,422 | Using config file: samconfig.toml, config environment: default 2022-07-07 14:37:13,422 | Expand command line arguments to: 2022-07-07 14:37:13,422 | --template_file=/tmp/aws-toolkit-vscode/vsctkB5jyaT/appvsctktemplate.yaml --build_dir=/tmp/aws-toolkit-vscode/vsctkB5jyaT/output --base_dir=/Users/fabian/Projects/sanfl-backend-api/src --cache_dir=.aws-sam/cache `
Additionally I've tried adding this to my launch.json configurations:
"sam": { "buildArguments": ["--cache-dir=/Users/fabian/aws-temp/cache"] }
which seems to correctly update the argument supplied to sam build, but .aws-sam directory is still trying to be accessed for some reason and failing:
`2022-07-07 14:54:44 [INFO]: Building SAM application... 2022-07-07 14:54:44 [INFO]: Command: (not started) [/usr/local/bin/sam build --debug --build-dir /tmp/aws-toolkit-vscode/vsctkoZCTH6/output --template /tmp/aws-toolkit-vscode/vsctkoZCTH6/appvsctktemplate.yaml --base-dir /Users/fabian/Projects/sanfl-backend-api/src --cache-dir=/Users/fabian/aws-temp/cache] 2022-07-07 14:54:44 [INFO]: 2022-07-07 14:54:44,916 | Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2022-07-07 14:54:44 [INFO]: 2022-07-07 14:54:44,916 | Using config file: samconfig.toml, config environment: default 2022-07-07 14:54:44,917 | Expand command line arguments to: 2022-07-07 14:54:44,917 | --template_file=/tmp/aws-toolkit-vscode/vsctkoZCTH6/appvsctktemplate.yaml --build_dir=/tmp/aws-toolkit-vscode/vsctkoZCTH6/output --base_dir=/Users/fabian/Projects/sanfl-backend-api/src --cache_dir=/Users/fabian/aws-temp/cache
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,345 | 'build' command is called
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,348 | No Parameters detected in the template
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,370 | There is no customer defined id or cdk path defined for resource src, so we will use the resource logical id as the resource id
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,370 | 0 stacks found in the template 2022-07-07 14:54:45,371 | No Parameters detected in the template
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,386 | There is no customer defined id or cdk path defined for resource src, so we will use the resource logical id as the resource id
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,386 | 1 resources found in the stack 2022-07-07 14:54:45,386 | Found Serverless function with name='src' and CodeUri='/Users/fabian/Projects/sanfl-backend-api/src'
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,389 | Instantiating build definitions
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,389 | No previous build graph found, generating new one 2022-07-07 14:54:45,390 | Unique function build definition found, adding as new (Function Build Definition: BuildDefinition(nodejs14.x, /Users/fabian/Projects/sanfl-backend-api/src, Zip, , 01415133-f893-4f66-9381-250609ddfc24, {}, {}, x86_64, []), Function: Function(function_id='src', name='src', functionname='src', runtime='nodejs14.x', memory=None, timeout=None, handler='functionTests/test_reprocessCurrentSeasonPlayerStats.test_reprocessCurrentSeasonPlayerStats', imageuri=None, packagetype='Zip', imageconfig=None, codeuri='/Users/fabian/Projects/sanfl-backend-api/src', environment={'Variables': OrderedDict()}, rolearn=None, layers=[], events=None, metadata={'SamResourceId': 'src'}, inlinecode=None, codesign_config_arn=None, architectures=None, function_url_config=None, stack_path=''))
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,391 | Building codeuri: /Users/fabian/Projects/sanfl-backend-api/src runtime: nodejs14.x metadata: {} architecture: x86_64 functions: src
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,391 | Building to following folder /private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,397 | Loading workflow module 'aws_lambda_builders.workflows'
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,400 | Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,402 | Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,405 | Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,407 | Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,410 | Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,412 | Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,414 | Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,416 | Registering workflow 'CustomMakeBuilder' with capability 'Capability(language='provided', dependency_manager=None, application_framework=None)'
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,418 | Registering workflow 'NodejsNpmEsbuildBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm-esbuild', application_framework=None)'
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,418 | Found workflow 'NodejsNpmBuilder' to support capabilities 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,419 | Running workflow 'NodejsNpmBuilder'
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,419 | Running NodejsNpmBuilder:NpmPack 2022-07-07 14:54:45,419 | NODEJS packaging file:/Users/fabian/Projects/sanfl-backend-api/src to /var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk 2022-07-07 14:54:45,419 | executing NPM: ['npm', 'pack', '-q', 'file:/Users/fabian/Projects/sanfl-backend-api/src']
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,980 | NODEJS packed to common-libraries-0.0.1.tgz
2022-07-07 14:54:45 [INFO]: 2022-07-07 14:54:45,982 | NODEJS extracting to /var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked
2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,016 | NodejsNpmBuilder:NpmPack succeeded
2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,017 | Running NodejsNpmBuilder:CopyNpmrcAndLockfile 2022-07-07 14:54:46,017 | package-lock.json copying in: /var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked/package
2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,018 | NodejsNpmBuilder:CopyNpmrcAndLockfile succeeded
2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,018 | Running NodejsNpmBuilder:CopySource 2022-07-07 14:54:46,018 | Creating target folders at /private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src
2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,018 | Copying directory metadata from source (/var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked/package) to destination (/private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src)
2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,019 | Creating target folders at /private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src/tests
2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,019 | Copying directory metadata from source (/var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked/package/tests) to destination (/private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src/tests)
2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,019 | Copying source file (/var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked/package/tests/test_appNews.js) to destination (/private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src/tests/test_appNews.js)
2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,020 | Copying source file (/var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked/package/tests/test_clubAds.js) to destination (/private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src/tests/test_clubAds.js)
...
2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,066 | Copying source file (/var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked/package/data/fixtureMethods.js) to destination (/private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src/data/fixtureMethods.js)
2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,067 | Copying source file (/var/folders/c1/j0pl0xn13_7g805y_phy2drr0000gn/T/tmp6q0fsfvk/unpacked/package/data/playerMethods.js) to destination (/private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src/data/playerMethods.js)
2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,067 | NodejsNpmBuilder:CopySource succeeded
2022-07-07 14:54:46 [INFO]: 2022-07-07 14:54:46,068 | Running NodejsNpmBuilder:NpmInstall 2022-07-07 14:54:46,068 | NODEJS installing in: /private/tmp/aws-toolkit-vscode/vsctkoZCTH6/output/src 2022-07-07 14:54:46,068 | executing NPM: ['npm', 'install', '-q', '--no-audit', '--no-save', '--production', '--unsafe-perm']
2022-07-07 14:54:47 [INFO]: 2022-07-07 14:54:47,618 | NodejsNpmBuilder:NpmInstall succeeded
2022-07-07 14:54:47 [INFO]: 2022-07-07 14:54:47,619 | Running NodejsNpmBuilder:CleanUp 2022-07-07 14:54:47,619 | Clean up action: .aws-sam/deps/01415133-f893-4f66-9381-250609ddfc24 does not exist and will be skipped. 2022-07-07 14:54:47,619 | NodejsNpmBuilder:CleanUp succeeded 2022-07-07 14:54:47,619 | Running NodejsNpmBuilder:CopyDependencies
2022-07-07 14:54:47 [INFO]: 2022-07-07 14:54:47,620 | Creating target folders at .aws-sam/deps/01415133-f893-4f66-9381-250609ddfc24/node_modules
2022-07-07 14:54:47 [INFO]: 2022-07-07 14:54:47,620 | NodejsNpmBuilder:CopyDependencies raised unhandled exception Traceback (most recent call last): File "/usr/local/Cellar/aws-sam-cli/1.53.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/workflow.py", line 301, in run action.execute() File "/usr/local/Cellar/aws-sam-cli/1.53.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/actions.py", line 132, in execute copytree(dependencies_source, new_destination) File "/usr/local/Cellar/aws-sam-cli/1.53.0/libexec/lib/python3.8/site-packages/aws_lambda_builders/utils.py", line 46, in copytree os.makedirs(destination) File "/usr/local/Cellar/python@3.8/3.8.13_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py", line 213, in makedirs makedirs(head, exist_ok=exist_ok) File "/usr/local/Cellar/python@3.8/3.8.13_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py", line 213, in makedirs makedirs(head, exist_ok=exist_ok) File "/usr/local/Cellar/python@3.8/3.8.13_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py", line 213, in makedirs makedirs(head, exist_ok=exist_ok) File "/usr/local/Cellar/python@3.8/3.8.13_1/Frameworks/Python.framework/Versions/3.8/lib/python3.8/os.py", line 223, in makedirs mkdir(name, mode) OSError: [Errno 30] Read-only file system: '.aws-sam'
2022-07-07 14:54:47 [INFO]: Build Failed
2022-07-07 14:54:47 [INFO]: 2022-07-07 14:54:47,633 | Sending Telemetry: {'metrics': [{'commandRun': {'requestId': 'e8325380-45a9-4768-bd3b-9d8fd9bef8de', 'installationId': 'da55d10c-476a-4985-893e-5e6fa7142ad5', 'sessionId': 'ebfb26b0-5f89-46fb-917e-d633a6c7b6fc', 'executionEnvironment': 'CLI', 'ci': False, 'pyversion': '3.8.13', 'samcliVersion': '1.53.0', 'awsProfileProvided': False, 'debugFlagProvided': True, 'region': '', 'commandName': 'sam build', 'metricSpecificAttributes': {'projectType': 'CFN'}, 'duration': 2716, 'exitReason': 'WorkflowUnknownError', 'exitCode': 1}}]}
2022-07-07 14:54:49 [INFO]: 2022-07-07 14:54:49,411 | HTTPSConnectionPool(host='aws-serverless-tools-telemetry.us-west-2.amazonaws.com', port=443): Read timed out. (read timeout=0.1)
2022-07-07 14:54:49 [INFO]: Error: NodejsNpmBuilder:CopyDependencies - [Errno 30] Read-only file system: '.aws-sam'
2022-07-07 14:54:49 [WARN]: "sam build" failed: /tmp/aws-toolkit-vscode/vsctkoZCTH6/appvsctktemplate.yaml`
Currently, I think these problems are because of SAM CLI's reliance on the current working directory for significant portions of its config resolution logic. The Toolkit does not provide an explicit working directory which can lead to all sorts of weird issues, especially when combined with the generated temporary directory (see https://github.com/aws/aws-toolkit-vscode/issues/2050).
The solution could be as simple as setting the current working directory to the SAM project, though this may have unintended consequences. Work is in progress for some sort of fix.
@JadenSimon
For my issue (that I wrongly posted here) that's somewhat related to this one, I just wanted to let you know that rolling back to aws-sam-cli 1.52.0
fixed the issue for me. Thanks!
https://github.com/aws/aws-toolkit-vscode/pull/2755 is a potential solution though it would be very helpful if anyone experiencing this problem could try out the fix:
Extensions: Install from VSIX...
command to install the VSIX.
Installing the above VSIX fixes the issue for me while using the latest version of AWS SAM CLI (v1.53.0). Thank you!
@JadenSimon
I too confirm that the above VSIX allows me to, once again, successfully use debugging with aws-sam-cli v1.53.0
. Thanks!
I take it this AWS Toolkit preview will simply be overwritten once the official 1.46.0
version is released? Thanks!
I take it this AWS Toolkit preview will simply be overwritten once the official 1.46.0 version is released? Thanks!
Yeah it should, at least from my experience. If it doesn't do it automatically then you can manually upgrade using the extensions tab.
I also can confirm this VSIX works for me. It did initially complain the ~/.aws-sam directory was read only so I deleted it manually then it worked.
This VSIX worked for me as well with aws-sam-cli 1.53.0. Thanks!
AWS Toolkit 1.46 includes a fix for this.
Describe the bug
When use VS Command
Deploy SAM Application
with dependencies layer, build status isfailed
.However, deploy process was continuing and create insufficient layer object(only requirements.txt file)
execution log is in
additional context
.failed reason is
[Errno 30] Read-only file system: '.aws-sam'
.when run command without AWS-ToolKit as below, dependencies ware resolved and create layer.
template.yml
my_layer/requirements.txt
has only line:requests
To Reproduce
Steps to reproduce the behavior:
requirements.txt
in layer directoryDeploy SAM Application
Expected behavior Build requirements.txt dependencies and create layer
Desktop (please complete the following information):
Additional context
sam_deploy.log