Some questions

[sammefford]

Let me know if this is the wrong place to put comments & questions.

First, awesome repo, I love what you're doing here! This comes close to answering my question of how to do this. I do have a few questions remaining:

1. why tensorflow 1.8 rather than 1.12?
2. why add keras separately? Does it have something tf.keras does not? Does it add much size to the layer?
3. does test.py need more meat in it? I notice in aws-build-lambda.sh it says "Now we run the classification on two images" but test.py doesn't appear to do that.

Also, any communities out there you use to collaborate with others trying to do similar things? We're trying to build a lambda which talks to TF Serving via grpc, but just to do that seems to require the tensorflow dependency which version 1.12 appears too large for a lambda.

[antonpaquin]

Hi Sam -- I'm glad you find it useful!

1. IIRC, 1.8 is the latest that fits under the 250MB limit. At some point I'd like to go back run a setup for each version that fits, but I haven't gotten around to it yet.
2. I looked up the difference and apparently it's only theano support. That might be a good way to strip down some size, I'll test it out to see if it works. Keras is about 40MB
3. This is a re-package of an older repo of mine, so some of the comments are off. Even without the session, import tensorflow on its own touches the main binary. I'm not 100% sure I'm not missing a hidden import somewhere, but it seems to be mostly functional.

I started this project with ideas from https://ryan-cranfill.github.io/keras-aws-lambda/ but besides that I don't know any others working on this issue.

I don't know if there's an easy solution to use a later version.

• Removing keras might give you a little bit of room to play with for later versions
• It might be possible to dynamically install what's missing by downloading to /tmp, but if you're not keeping your lambda warm that will be a lot of latency on the first call.
• If you're not too tied to AWS, I believe azure's faas doesn't have a size limit
• Lambda's concurrency model is to just spawn a new lambda, which removes the need for tf serving in some cases

Let me know if you find something that works.

[danielmorvay]

Hi. Nice layer. The combination what I'm looking for is the PIL, numpy, keras and TF in one pack fits into the 262MBi limit. Without PIL this package hardly (does not?) work for image recognition. Or it does, I just did not manage to hack it. :) Please let me know if you know the solution. Or will you consider to add PIL to the package? (also might be interesting: p2.7) All in all, nice pack. Congrat. BRD.

[antonpaquin]

Hi Daniel, I've added a PIL layer (ARN is in the README). With TF 1.8.0 it fits into about 200MB, as you get more recent it becomes harder to fit. I've also added a way to make new layers pretty easy to generate (as long as it's python 3.6.6) I hope this helps, let me know if you notice any strangeness.

[danielmorvay]

Hi. In the meantime I've found and fine tuned a python2.7 based layer (custom env) with older keras and TF, so your pack which includes newer releases seems to be better. Nevertheless I have issues with importing your packs as layer, it seems that the authorization needs further config on your side. I got the following alert when I tried to save my lambda using any of your packs.
"You are not authorized to perform: lambda:GetLayerVersion." If you have only one AWS account I can help you testing the authorization config. (OK/NOK) BRD.

[antonpaquin]

That's interesting: I've got

  "Version": "2012-10-17",
  "Id": "default",
  "Statement": [
    {
      "Sid": "publish",
      "Effect": "Allow",
      "Principal": "*",
      "Action": "lambda:GetLayerVersion",
      "Resource": "arn:aws:lambda:us-east-1:347034527139:layer:tf_keras_pillow:3"
    }
  ]
}

permissions set.

I wonder: do I need to make the objects public on s3? I've done that, let me know if it works.

Happy to have the help if you're willing.

[danielmorvay]

Hi. 2 things might be missing:

1. grant users outside of us-east-1 access to this arn. (use wildcard). I'm using mainly mainly deployment in Ireland.
2. publish the layer.

check this out: https://docs.aws.amazon.com/lambda/latest/dg/lambda-api-permissions-ref.html#permissions-resources-layers BRD.

[danielmorvay]

Wow. My guess was 50% accurate only. :) It is working from us-east-1. I just tested it now from N. Virginia (us-east-1) and I managed to save the config with your layer under my "hello world" code. So if you didn't change anything in the meantime, using wildcard instead of us-east-1 might help by itself. I hope I helped.

[yurylitvinov]

Hi there. Just FYI: a couple of weeks ago I got the message "You are not authorized to perform: lambda:GetLayerVersion." when trying to change layer version number or use ARN without version number. When using original ARN everything was fine (from N. Virginia region). So may be there is no issue with permissions on Anton side.

On another matter: Anton, did you try to use load_model function from keras with your package? I could not load a keras model and I wonder if the problem was that I saved model from different version of keras or that your layer is missing something for loading models.

Thanks! Yuri.

[antonpaquin]

I'm following the steps outlined here to publish the layers. Example command:

aws lambda add-layer-version-permission --layer-name tf_1_12_keras --version-number 1 --principal "*" --statement-id publish --action lambda:GetLayerVersion

This doesn't say anything about regions. If it's failing outside us-east-1, my guess is that layers aren't cross region, and I'll have to publish a version for each region. This is fine, I'll write a script to run that process hopefully tonight.

[antonpaquin]

Yuri, I've created a new issue #4 for the load_model issue.

[antonpaquin]

I've deployed the layer to all other regions I can get to (which excludes us-gov-* and cn-*)

(This also involves scripting the deployment, since it's a bit much to do manually).

You should find links to the tables in the ReadMe. Let me know if it works, or if you're still having issues.