aws / aws-lambda-builders

Python library to compile, build & package AWS Lambda functions for several runtimes & framework
Apache License 2.0
334 stars 139 forks source link

Build via `sam build` Fails When "Amazon.Lambda.Tools" Is Installed in Built Docker Container #126

Open chrisoverzero opened 5 years ago

chrisoverzero commented 5 years ago

Description

When using a Docker image which has the dotnet tool "Amazon.Lambda.Tools" already installed, the command sam build will fail when trying to update the tool. It's something about hardlinks which I'm not knowledgeable enough about Docker to speculate on.

Steps to reproduce

As documented in the example repository I created.

Observed result

Behind the fold. ``` 2019-08-20 20:28:18 Using SAM Template at /var/task/template.yml 2019-08-20 20:28:18 Changing event name from creating-client-class.iot-data to creating -client-class.iot-data-plane 2019-08-20 20:28:18 Changing event name from before-call.apigateway to before-call.api- gateway 2019-08-20 20:28:18 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict 2019-08-20 20:28:18 Changing event name from before-parameter-build.autoscaling.CreateL aunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration 2019-08-20 20:28:18 Changing event name from before-parameter-build.route53 to before-p arameter-build.route-53 2019-08-20 20:28:18 Changing event name from request-created.cloudsearchdomain.Search t o request-created.cloudsearch-domain.Search 2019-08-20 20:28:18 Changing event name from docs.*.autoscaling.CreateLaunchConfigurati on.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section 2019-08-20 20:28:18 Changing event name from before-parameter-build.cloudsearchdomain.S earch to before-parameter-build.cloudsearch-domain.Search 2019-08-20 20:28:18 Changing event name from docs.*.cloudsearchdomain.Search.complete-s ection to docs.*.cloudsearch-domain.Search.complete-section 2019-08-20 20:28:18 Changing event name from before-parameter-build.logs.CreateExportTa sk to before-parameter-build.cloudwatch-logs.CreateExportTask 2019-08-20 20:28:18 Changing event name from docs.*.logs.CreateExportTask.complete-sect ion to docs.*.cloudwatch-logs.CreateExportTask.complete-section 2019-08-20 20:28:18 Changing event name from creating-client-class.iot-data to creating -client-class.iot-data-plane 2019-08-20 20:28:18 Changing event name from before-call.apigateway to before-call.api- gateway 2019-08-20 20:28:18 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict 2019-08-20 20:28:18 Changing event name from before-parameter-build.autoscaling.CreateL aunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration 2019-08-20 20:28:18 Changing event name from before-parameter-build.route53 to before-p arameter-build.route-53 2019-08-20 20:28:18 Changing event name from request-created.cloudsearchdomain.Search t o request-created.cloudsearch-domain.Search 2019-08-20 20:28:18 Changing event name from docs.*.autoscaling.CreateLaunchConfigurati on.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section 2019-08-20 20:28:18 Changing event name from before-parameter-build.cloudsearchdomain.S earch to before-parameter-build.cloudsearch-domain.Search 2019-08-20 20:28:18 Changing event name from docs.*.cloudsearchdomain.Search.complete-s ection to docs.*.cloudsearch-domain.Search.complete-section 2019-08-20 20:28:18 Changing event name from before-parameter-build.logs.CreateExportTa sk to before-parameter-build.cloudwatch-logs.CreateExportTask 2019-08-20 20:28:18 Changing event name from docs.*.logs.CreateExportTask.complete-sect ion to docs.*.cloudwatch-logs.CreateExportTask.complete-section 2019-08-20 20:28:18 'build' command is called 2019-08-20 20:28:18 No Parameters detected in the template 2019-08-20 20:28:18 1 resources found in the template 2019-08-20 20:28:18 Found Serverless function with name='Example' and CodeUri='./src/Ex ample' 2019-08-20 20:28:18 Building resource 'Example' 2019-08-20 20:28:18 Loading workflow module 'aws_lambda_builders.workflows' 2019-08-20 20:28:18 Registering workflow 'PythonPipBuilder' with capability 'Capability (language='python', dependency_manager='pip', application_framework=None)' 2019-08-20 20:28:18 Registering workflow 'NodejsNpmBuilder' with capability 'Capability (language='nodejs', dependency_manager='npm', application_framework=None)' 2019-08-20 20:28:18 Registering workflow 'RubyBundlerBuilder' with capability 'Capabili ty(language='ruby', dependency_manager='bundler', application_framework=None)' 2019-08-20 20:28:18 Registering workflow 'GoDepBuilder' with capability 'Capability(lan guage='go', dependency_manager='dep', application_framework=None)' 2019-08-20 20:28:18 Registering workflow 'GoModulesBuilder' with capability 'Capability (language='go', dependency_manager='modules', application_framework=None)' 2019-08-20 20:28:18 Registering workflow 'JavaGradleWorkflow' with capability 'Capabili ty(language='java', dependency_manager='gradle', application_framework=None)' 2019-08-20 20:28:18 Registering workflow 'JavaMavenWorkflow' with capability 'Capabilit y(language='java', dependency_manager='maven', application_framework=None)' 2019-08-20 20:28:18 Registering workflow 'DotnetCliPackageBuilder' with capability 'Cap ability(language='dotnet', dependency_manager='cli-package', application_framework=None )' 2019-08-20 20:28:18 Found workflow 'DotnetCliPackageBuilder' to support capabilities 'C apability(language='dotnet', dependency_manager='cli-package', application_framework=No ne)' 2019-08-20 20:28:18 Running workflow 'DotnetCliPackageBuilder' 2019-08-20 20:28:18 Running DotnetCliPackageBuilder:GlobalToolInstall 2019-08-20 20:28:18 Installing Amazon.Lambda.Tools Global Tool 2019-08-20 20:28:18 executing dotnet: ['dotnet', 'tool', 'install', '-g', 'Amazon.Lambd a.Tools'] 2019-08-20 20:28:18 2019-08-20 20:28:18 Error installing probably due to already installed. Attempt to upda te to latest version. 2019-08-20 20:28:18 executing dotnet: ['dotnet', 'tool', 'update', '-g', 'Amazon.Lambda .Tools'] 2019-08-20 20:28:19 2019-08-20 20:28:19 DotnetCliPackageBuilder:GlobalToolInstall failed Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/aws_lambda_builders/workflow.py", line 2 48, in run action.execute() File "/usr/local/lib/python2.7/site-packages/aws_lambda_builders/workflows/dotnet_cli package/actions.py", line 42, in execute raise ActionFailedError("Error configuring the Amazon.Lambda.Tools .NET Core Global Tool: " + str(ex)) ActionFailedError: Error configuring the Amazon.Lambda.Tools .NET Core Global Tool: Dot net CLI Failed: Tool 'amazon.lambda.tools' failed to update due to the following: Failed to uninstall tool package 'amazon.lambda.tools': Invalid cross-device link Build Failed Error: DotnetCliPackageBuilder:GlobalToolInstall - Error configuring the Amazon.Lambda. Tools .NET Core Global Tool: Dotnet CLI Failed: Tool 'amazon.lambda.tools' failed to up date due to the following: Failed to uninstall tool package 'amazon.lambda.tools': Invalid cross-device link ```

Expected result

A successful build. Whether that's via a successful update of the tool or via not trying to update the tool (or some other thing?), I have no opinion.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

  1. OS: In the example case, Amazon Linux.
  2. sam --version: SAM CLI, version 0.19.0

Add --debug flag to command you are running

chrisoverzero commented 5 years ago

I wonder if treating the user's installation of dotnet tools as read-write isn't a suboptimal idea all around. Given the installation command as:

self.subprocess_dotnet.run(
    ['tool', 'install', '-g', 'Amazon.Lambda.Tools'],
)

...it could become something like:

self.subprocess_dotnet.run(
    ['tool', 'install', '--tool-path', './.aws-sam/.dotnet/', 'Amazon.Lambda.Tools'],
)

...though that's not a concrete proposal. I'm sure some attribute of self (artifacts_dir?) has the ".aws-sam/" value, and there may be a better reserved name than ".dotnet".

Then the "package" command would reach into that "${--build-dir}/.dotnet/" directory for its local copy of "dotnet-lambda".