search

aws / aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
https://aws.amazon.com/serverless/sam/
Apache License 2.0
6.5k stars 1.17k forks source link

sam build redownloads all Python dependencies #1329

Closed Mike-Nahmias closed 5 years ago

Mike-Nahmias commented 5 years ago

Description

I was starting to create a feature request for an option to run sam build and skip dependencies due to the amount of time it takes. However, it seems that the larger issue is PythonPipBuilder redownloading everything even though dependencies aren't changing. Every time I make a change to a function and want to test it again I have to run sam build. Most of the time spent is building Python dependencies even though none of them changed. I'd say 99% of the time when I run sam build it's just to update the code.

Steps to reproduce

Observed result

2019-08-09 12:08:16 Using SAM Template at c:\Users\mnahmias\PycharmProjects\sam-app\template.yaml
2019-08-09 12:08:16 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-08-09 12:08:16 Changing event name from before-call.apigateway to before-call.api-gateway
2019-08-09 12:08:16 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-08-09 12:08:16 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-08-09 12:08:16 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-08-09 12:08:16 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-08-09 12:08:16 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-08-09 12:08:16 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-08-09 12:08:16 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-08-09 12:08:16 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-08-09 12:08:16 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-08-09 12:08:16 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-08-09 12:08:16 Changing event name from before-call.apigateway to before-call.api-gateway
2019-08-09 12:08:16 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-08-09 12:08:16 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-08-09 12:08:16 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-08-09 12:08:16 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-08-09 12:08:16 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-08-09 12:08:16 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-08-09 12:08:16 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-08-09 12:08:16 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-08-09 12:08:16 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-08-09 12:08:16 Telemetry endpoint configured to be https://aws-serverless-tools-telemetry.us-west-2.amazonaws.com/metrics
2019-08-09 12:08:16 'build' command is called
2019-08-09 12:08:16 No Parameters detected in the template
2019-08-09 12:08:16 2 resources found in the template
2019-08-09 12:08:16 Found Serverless function with name='HelloWorldFunction' and CodeUri='hello_world/'
2019-08-09 12:08:16 Building resource 'HelloWorldFunction'
2019-08-09 12:08:16 Loading workflow module 'aws_lambda_builders.workflows'
2019-08-09 12:08:16 Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
2019-08-09 12:08:16 Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
2019-08-09 12:08:16 Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
2019-08-09 12:08:16 Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
2019-08-09 12:08:16 Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
2019-08-09 12:08:17 Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
2019-08-09 12:08:17 Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
2019-08-09 12:08:17 Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
2019-08-09 12:08:17 Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
2019-08-09 12:08:17 Running workflow 'PythonPipBuilder'
2019-08-09 12:08:17 Running PythonPipBuilder:ResolveDependencies
2019-08-09 12:08:17 calling pip download -r C:\Users\mnahmias\OneDrive - Bay State College\PycharmProjects\sam-app\hello_world\requirements.txt --dest C:\Users\mnahmias\AppData\Local\Temp\tmp7kih70k1
2019-08-09 12:08:20 Full dependency closure: {chardet==3.0.4(wheel), requests==2.22.0(wheel), idna==2.8(wheel), urllib3==1.25.3(wheel), certifi==2019.6.16(wheel)}
2019-08-09 12:08:20 initial compatible: {chardet==3.0.4(wheel), requests==2.22.0(wheel), idna==2.8(wheel), urllib3==1.25.3(wheel), certifi==2019.6.16(wheel)}
2019-08-09 12:08:20 initial incompatible: set()
2019-08-09 12:08:20 Downloading missing wheels: set()
2019-08-09 12:08:20 compatible wheels after second download pass: {chardet==3.0.4(wheel), requests==2.22.0(wheel), idna==2.8(wheel), urllib3==1.25.3(wheel), certifi==2019.6.16(wheel)}
2019-08-09 12:08:20 Build missing wheels from sdists (C compiling True): set()
2019-08-09 12:08:20 compatible after building wheels (no C compiling): {chardet==3.0.4(wheel), requests==2.22.0(wheel), idna==2.8(wheel), urllib3==1.25.3(wheel), certifi==2019.6.16(wheel)}
2019-08-09 12:08:20 Build missing wheels from sdists (C compiling False): set()
2019-08-09 12:08:20 compatible after building wheels (C compiling): {chardet==3.0.4(wheel), requests==2.22.0(wheel), idna==2.8(wheel), urllib3==1.25.3(wheel), certifi==2019.6.16(wheel)}
2019-08-09 12:08:20 Final compatible: {urllib3==1.25.3(wheel), chardet==3.0.4(wheel), requests==2.22.0(wheel), idna==2.8(wheel), certifi==2019.6.16(wheel)}
2019-08-09 12:08:20 Final incompatible: set()
2019-08-09 12:08:20 Final missing wheels: set()
2019-08-09 12:08:21 PythonPipBuilder:ResolveDependencies succeeded
2019-08-09 12:08:21 Running PythonPipBuilder:CopySource
2019-08-09 12:08:21 PythonPipBuilder:CopySource succeeded

Build Succeeded

Expected result

I would expect it to see that all requirements are already met.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

  1. OS: Windows 10 1903
  2. sam --version: 0.19.0
jfuss commented 5 years ago

@Mike-Nahmias Thanks for the report. The current behavior is a fresh/clean build every time. The issue with incremental builds is how we will support this within the container (when --use-container option is passed). Not saying this isn't an issue but one of the reasons why we started with clean builds.

We already have an open issue for this request (#805). So closing this in favor of the one already open. Feel free to add further thoughts and comment there.

Closing as this is a duplicate