search

aws / aws-sam-cli

CLI tool to build, test, debug, and deploy Serverless applications using AWS SAM
https://aws.amazon.com/serverless/sam/
Apache License 2.0
6.5k stars 1.17k forks source link

Ubuntu WSL error on 'sam build': Error: PythonPipBuilder:CopySource - name 'WindowsError' is not defined #1195

Closed johnc44 closed 4 years ago

johnc44 commented 5 years ago

Description

When using Ubuntu WSL on Windows 10, running "sam build" on Windows mounted drive on a python3.6 lambda which has a subfolder causes the error: Error: PythonPipBuilder:CopySource - name 'WindowsError' is not defined

Steps to reproduce

I have Ubuntu WSL configured so that "/c" goes to my Windows C drive (instead of the usual /mnt/c). This was to get around another SAM/Docker problem. But I was getting the same issue before I did this.

Anyway, from Ubuntu shell in Win10:

cd /c/temp
sam init --runtime python3.6 -o wsltest2
cd wsltest2/sam-app
sam build
# this works
mkdir hello_world/test
sam build
# this fails

Observed result

2019-05-24 15:13:15 Using SAM Template at /c/temp/wsltest2/sam-app/template.yaml
2019-05-24 15:13:15 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-05-24 15:13:15 Changing event name from before-call.apigateway to before-call.api-gateway
2019-05-24 15:13:15 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-05-24 15:13:15 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-05-24 15:13:15 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-05-24 15:13:15 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-05-24 15:13:15 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-05-24 15:13:15 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-05-24 15:13:15 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-05-24 15:13:15 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-05-24 15:13:15 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-05-24 15:13:15 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-05-24 15:13:15 Changing event name from before-call.apigateway to before-call.api-gateway
2019-05-24 15:13:15 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-05-24 15:13:15 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-05-24 15:13:15 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-05-24 15:13:15 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-05-24 15:13:15 Changing event name from docs.*.autoscaling.CreateLaunchConfiguration.complete-section to docs.*.auto-scaling.CreateLaunchConfiguration.complete-section
2019-05-24 15:13:15 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-05-24 15:13:15 Changing event name from docs.*.logs.CreateExportTask.complete-section to docs.*.cloudwatch-logs.CreateExportTask.complete-section
2019-05-24 15:13:15 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-05-24 15:13:15 Changing event name from docs.*.cloudsearchdomain.Search.complete-section to docs.*.cloudsearch-domain.Search.complete-section
2019-05-24 15:13:15 'build' command is called
2019-05-24 15:13:15 No Parameters detected in the template
2019-05-24 15:13:15 2 resources found in the template
2019-05-24 15:13:15 Found Serverless function with name='HelloWorldFunction' and CodeUri='hello_world/'
2019-05-24 15:13:15 Building resource 'HelloWorldFunction'
2019-05-24 15:13:15 Loading workflow module 'aws_lambda_builders.workflows'
2019-05-24 15:13:15 Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
2019-05-24 15:13:15 Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
2019-05-24 15:13:15 Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
2019-05-24 15:13:15 Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
2019-05-24 15:13:15 Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
2019-05-24 15:13:15 Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
2019-05-24 15:13:15 Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
2019-05-24 15:13:15 Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
2019-05-24 15:13:15 Found workflow 'PythonPipBuilder' to support capabilities 'Capability(language='python', dependency_manager='pip', application_framework=None)'
2019-05-24 15:13:15 Running workflow 'PythonPipBuilder'
2019-05-24 15:13:15 Running PythonPipBuilder:ResolveDependencies
2019-05-24 15:13:16 calling pip download -r /c/temp/wsltest2/sam-app/hello_world/requirements.txt --dest /tmp/tmpvb9pkjda
2019-05-24 15:13:18 Full dependency closure: {requests==2.22.0(wheel), chardet==3.0.4(wheel), idna==2.8(wheel), urllib3==1.25.3(wheel), certifi==2019.3.9(wheel)}
2019-05-24 15:13:18 initial compatible: {requests==2.22.0(wheel), chardet==3.0.4(wheel), idna==2.8(wheel), urllib3==1.25.3(wheel), certifi==2019.3.9(wheel)}
2019-05-24 15:13:18 initial incompatible: set()
2019-05-24 15:13:18 Downloading missing wheels: set()
2019-05-24 15:13:18 compatible wheels after second download pass: {requests==2.22.0(wheel), chardet==3.0.4(wheel), idna==2.8(wheel), urllib3==1.25.3(wheel), certifi==2019.3.9(wheel)}
2019-05-24 15:13:18 Build missing wheels from sdists (C compiling True): set()
2019-05-24 15:13:18 compatible after building wheels (no C compiling): {requests==2.22.0(wheel), chardet==3.0.4(wheel), idna==2.8(wheel), urllib3==1.25.3(wheel), certifi==2019.3.9(wheel)}
2019-05-24 15:13:18 Build missing wheels from sdists (C compiling False): set()
2019-05-24 15:13:18 compatible after building wheels (C compiling): {requests==2.22.0(wheel), chardet==3.0.4(wheel), idna==2.8(wheel), urllib3==1.25.3(wheel), certifi==2019.3.9(wheel)}
2019-05-24 15:13:18 Final compatible: {requests==2.22.0(wheel), chardet==3.0.4(wheel), idna==2.8(wheel), urllib3==1.25.3(wheel), certifi==2019.3.9(wheel)}
2019-05-24 15:13:18 Final incompatible: set()
2019-05-24 15:13:18 Final missing wheels: set()
2019-05-24 15:13:19 PythonPipBuilder:ResolveDependencies succeeded
2019-05-24 15:13:19 Running PythonPipBuilder:CopySource
2019-05-24 15:13:19 PythonPipBuilder:CopySource raised unhandled exception
Traceback (most recent call last):
  File "/home/johnc/.local/lib/python3.6/site-packages/aws_lambda_builders/utils.py", line 38, in copytree
    shutil.copystat(source, destination)
  File "/usr/lib/python3.6/shutil.py", line 225, in copystat
    _copyxattr(src, dst, follow_symlinks=follow)
  File "/usr/lib/python3.6/shutil.py", line 165, in _copyxattr
    os.setxattr(dst, name, value, follow_symlinks=follow_symlinks)
PermissionError: [Errno 13] Permission denied: '/c/temp/wsltest2/sam-app/.aws-sam/build/HelloWorldFunction/sub'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/johnc/.local/lib/python3.6/site-packages/aws_lambda_builders/workflow.py", line 248, in run
    action.execute()
  File "/home/johnc/.local/lib/python3.6/site-packages/aws_lambda_builders/actions.py", line 101, in execute
    copytree(self.source_dir, self.dest_dir, ignore=shutil.ignore_patterns(*self.excludes))
  File "/home/johnc/.local/lib/python3.6/site-packages/aws_lambda_builders/utils.py", line 58, in copytree
    copytree(new_source, new_destination, ignore=ignore)
  File "/home/johnc/.local/lib/python3.6/site-packages/aws_lambda_builders/utils.py", line 39, in copytree
    except WindowsError as ex:  # pylint: disable=undefined-variable
NameError: name 'WindowsError' is not defined

Build Failed
Error: PythonPipBuilder:CopySource - name 'WindowsError' is not defined

Expected result

It should build!

If I do the exact same steps to reproduce outside of the Windows drive (e.g. using /tmp instead of /c/temp) then it all works fine.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

  1. OS: Win10
  2. sam --version:0.16.0

This is particularly frustrating because my IDE is in Windows, but I can't use git dependencies in requirements.txt. But I can in Ubuntu WSL. So I have to maintain 2 file locations and keep copying files over from Win to Linux.

sriram-mv commented 5 years ago

Where is /c/temp/wsltest2/sam-app/.aws-sam/build/HelloWorldFunction/sub coming from? Given that your CodeUri is hello_world. will try to reproduce this on windows.

2ps commented 5 years ago

This issue has to do with windows extended attributes (in particular, system.wsl_case_sensitive) flag that Microsoft added to wsl to enable per-directory case-sensitivity settings. To fix the issue, open up the folder in explorer, right click and go to properties, and go to the security tab. You will need to add advanced permissions as specified here:

Note that the per-directory case sensitivity flag is not inherited; directories created in a case sensitive directory are not automatically case sensitive themselves. You must explicitly mark each directory as case sensitive. Changing the flag requires “write attributes” permission to the directory.

sriram-mv commented 4 years ago

Looks like the question has been answered, closing this issue.