search

mozilla-services / remote-settings-lambdas

Remote Settings scheduled tasks
Apache License 2.0
2 stars 8 forks source link

Python 3.7 #252

Closed peterbe closed 5 years ago

peterbe commented 5 years ago

Not sure what the state of 3.7 support is inside the libraries we depend on. E.g. kinto-signer.

Whatever we do, we must remember that we have 2 different Python environments. The one inside Dockerfile and the one inside the .travis.yml file.

leplatrem commented 5 years ago

python 3.7 should be our target, that's what we use in kinto-dist (where kinto-signer is used).

However I don't know about AWS lambda. @autrilla any hint?

autrilla commented 5 years ago

Currently Lamda only supports up to 3.6 https://docs.aws.amazon.com/lambda/latest/dg/lambda-runtimes.html

peterbe commented 5 years ago

I don't understand. What about this:

Screen Shot 2019-03-13 at 8 31 36 AM
autrilla commented 5 years ago

I misread, you're right. We can use 3.7 with no issue.

peterbe commented 5 years ago

I see now that they way they sorted that table, it totally looks like 3.6 is the highest version and 2.7 the lowest. Silly of them.

peterbe commented 5 years ago

Which one should you use python3.7-slim or python3.7-alpine? I never know.

According to this blog post the difference between python3.7 and python3.7-alpine is 714MB vs 115MB respectively. But it doesn't mention python3.7-slim.

According to this blog post the Alpine one is, by 54% share, the most popular Python docker image.

Does it matter? Possibly not much because we only use Docker to generate the .zip which is uploaded it AWS Lambda. But perhaps when we move to GCP it starts to matter since we'd run the docker containers as serverless components. Right?

In the grand scheme of things, it probably doesn't matter much, but it'd be interesting to take this opportunity to learn for when it does matter.

autrilla commented 5 years ago

I don't think it matters right now for Lambda, but as you mention it will matter on GCP. I think alpine images weren't great security wise, at least in the past, so we lean towards not using those. I don't see a reason not to use -slim though.

peterbe commented 5 years ago

@mythmon Do you have any strong preferences or insights on the topic starting at comment: https://github.com/mozilla-services/remote-settings-lambdas/issues/252#issuecomment-472988663

peterbe commented 5 years ago

alpine images weren't great security wise

Seems like a big statement. Do you have any links to point to that?

mythmon commented 5 years ago

I don't have strong opinions, but I have some weak ones. I just checked, and the sizes are as follows:

Image Size
python:3.7 929MB
python:3.7-slim 143MB
python:3.7-alpine 86MB

-slim gets away with this size by not including lots of things. I think that's fine, as it is pretty easy to add them back in with apt-get. I think using -slim is an easy win that we should do.

-alpine makes this even smaller mostly by switching out Debian for Alpine Linux. There isn't anything wrong with Alpine that I know, but it has fewer users, fewer developers, and generally less attention. That means that any problems that exist (security or otherwise) will go unnoticed longer, and take longer to fix, on average.

I also don't know the Alpine userland very well. I feel much more comfortable in Debian. It has the tools I expect, and most resources online refer to Debian, not Alpine.

For both of these reasons, I think python:3.7-slim is the image we should use.

autrilla commented 5 years ago

alpine images weren't great security wise

Seems like a big statement. Do you have any links to point to that?

I was pointed to this: https://kubedex.com/follow-up-container-scanning-comparison/. Really, I have no idea what we should use, but if we want to use alpine I'd talk to secops beforehand.

peterbe commented 5 years ago

Our current Dockerfile contains: apt-get update && apt-get install -y zip which is another reason to not rock the Debian-boat.