Golang binary provided as a zip file does not work with v0.3.0

[aleccarper]

Description:

After upgrading to 0.3.0 I am now getting the following error while using sam local start-api

{
  "errorMessage": "fork/exec /var/task/main: permission denied",
  "errorType": "PathError"
}

I have tried rebuilding my functions using

    GOOS=linux go build -o main main.go
    zip -j main.zip main

but that doesn't seem to be fixing the issue.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc) Mac/OSX

Output of sam --version: SAM CLI, version 0.3.0

Optional Debug logs:

2018-05-08 21:40:21 http://localhost:None "POST /v1.35/images/create?tag=go1.x&fromImage=lambci%2Flambda HTTP/1.1" 200 None

Fetching lambci/lambda:go1.x Docker container image......
2018-05-08 21:40:21 Mounting /private/var/folders/6k/wl5glw6x5vb2s4z2f6yfqr280000gn/T/tmplyA5JV as /var/task:ro inside runtime container
2018-05-08 21:40:21 http://localhost:None "POST /v1.35/containers/create HTTP/1.1" 201 90
2018-05-08 21:40:21 http://localhost:None "GET /v1.35/containers/e0bbd05b31ca340043f139387946e89ac1fb69d0c9b32ff19d81e456d5522bc6/json HTTP/1.1" 200 None
2018-05-08 21:40:21 http://localhost:None "GET /v1.35/containers/e0bbd05b31ca340043f139387946e89ac1fb69d0c9b32ff19d81e456d5522bc6/json HTTP/1.1" 200 None
2018-05-08 21:40:22 http://localhost:None "POST /v1.35/containers/e0bbd05b31ca340043f139387946e89ac1fb69d0c9b32ff19d81e456d5522bc6/start HTTP/1.1" 204 0
2018-05-08 21:40:22 Starting a timer for 90 seconds for function 'RegionRates'
2018-05-08 21:40:22 http://localhost:None "GET /v1.35/containers/e0bbd05b31ca340043f139387946e89ac1fb69d0c9b32ff19d81e456d5522bc6/json HTTP/1.1" 200 None
2018-05-08 21:40:22 http://localhost:None "POST /containers/e0bbd05b31ca340043f139387946e89ac1fb69d0c9b32ff19d81e456d5522bc6/attach?stream=1&stdin=0&logs=1&stderr=1&stdout=1 HTTP/1.1" 101 0
START RequestId: 9c3b6799-610b-1d1a-9e28-e37afcec3d87 Version: $LATEST
END RequestId: 9c3b6799-610b-1d1a-9e28-e37afcec3d87
REPORT RequestId: 9c3b6799-610b-1d1a-9e28-e37afcec3d87  Duration: 1.43 ms   Billed Duration: 100 ms Memory Size: 128 MB Max Memory Used: 6 MB
{
  "errorMessage": "fork/exec /var/task/main: permission denied",
  "errorType": "PathError"
}
2018-05-08 21:40:22 http://localhost:None "GET /v1.35/containers/e0bbd05b31ca340043f139387946e89ac1fb69d0c9b32ff19d81e456d5522bc6/json HTTP/1.1" 200 None
2018-05-08 21:40:22 http://localhost:None "DELETE /v1.35/containers/e0bbd05b31ca340043f139387946e89ac1fb69d0c9b32ff19d81e456d5522bc6?force=True&link=False&v=False HTTP/1.1" 204 0
2018-05-08 21:40:22 Function returned an invalid response (must include one of: body, headers or statusCode in the response object). Response received:
2018-05-08 21:40:22 127.0.0.1 - - [08/May/2018 21:40:22] "POST /v2/exports/rates HTTP/1.1" 502 -

[sanathkr]

You should chmod and give your binary executable permissions

[nzoschke]

I'm also getting this on a Go project. My reproducible branch is at https://github.com/nzoschke/gofaas/pull/62

[sanathkr]

Turns out, this is a regression from previous version because Python's ZipFile module does not retain file permissions on unzipping - https://bugs.python.org/issue15795.

Several folks have fixed it in different ways (ex: https://www.burgundywall.com/post/preserving-file-perms-with-python-zipfile-module). We need to implement a clean solution to this problem.

We will prioritize this and get a release out asap

[nzoschke]

We did some digging in on the awsdevelopers Slack. With some sleep and debug statements I see that the tmp dir doesn't have any executable bits on the main file:

$ cd /var/folders/px/fd8j3qvn13gcxw9_nw25pphw0000gn/T/tmpKXhUgv
$ ls -al
total 36816
drwx------    3 noah  staff       102 May  9 07:45 .
drwx------@ 259 noah  staff      8806 May  9 07:45 ..
-rw-r--r--    1 noah  staff  18846338 May  9 07:45 main

@sanathkr found this, which points to Python zip library:

looks like zipfile Python module doesn’t retain permissions on unzip.. - https://stackoverflow.com/questions/434641/how-do-i-set-permissions-attributes-on-a-file-in-a-zip-file-using-pythons-zip/6297838#6297838

[sanathkr]

This problems is specifically only when you bundle the Golang binary as a zipfile and provide to CodeUri. As a workaround, you can set CodeUri to be a path to a folder that contains your binary. This will work. In fact this will be faster because SAM CLI does not have to unzip every time you run the invoke.

If you have something like:

MyFunction:
  Type: AWS::Serverless::Function
  Properties:
     ...
     CodeUri: ./myfunction.zip

Instead, convert it to something like:

MyFunction:
  Type: AWS::Serverless::Function
  Properties:
     ...
     CodeUri: ./build

Where the build folder contains your Golang binary. This will save you the step of building a zip file and also improve performance because SAM CLI doesn't have to unzip on every invoke.

[kriation]

This problems is specifically only when you bundle the Golang binary as a zipfile and provide to CodeUri.

I can confirm that this is an issue on Linux for a Python project (the issue is not limited to Golang on MacOS).

As a workaround, you can set CodeUri to be a path to a folder that contains your binary.

I just tested this, and it worked properly. Thanks!

[aleccarper]

Thanks all!

[jfuss]

Reopening. The root cause of this is not fixed.