Python Support

[LewisLebentz]

I see in the Dockerfile there is NO_PYTHON=1, I'm assuming this breaks Python support? Is there a reason for this?

How would I go about creating my own Layer that includes Git for use inside Python? I am aiming to use this library which requires git:

https://github.com/dxa4481/truffleHog

[mhart]

The reason is to reduce size – but I don't think it does what you think it does.

Near I can tell, GitPython (which truffleHog uses) just requires the git binary: https://github.com/gitpython-developers/GitPython#requirements – which this layer has.

Give it a shot!

[mhart]

Going to close this – if you actually get an error trying to call git from Python, feel free to post it here

[LewisLebentz]

Hi @mhart sorry I'm new to Layers, trying to understand how they work still!

Tried using the layer earlier, and without it, it runs - just failing at the part where it tries to use git. When I add the layer all I get is "Unable to import module 'handler'".

No idea how to troubleshoot it, if I remove the layer it runs 🤷‍♂️. Does the handler need to change when using a Layer?

[mhart]

Nope, shouldn't need to change at all. Might be that some sort of error is happening during the import process?

You could try using docker-lambda to replicate the error locally to help with debugging it.

Otherwise, sounds like you might just need to triage a little more.

[mhart]

Also, if you post a small reproduction, I can try to triage myself and see what's up

[LewisLebentz]

There isn't much to it right now, still just trying to get the basics working before developing it.

GitHub sends details of a commit to API Gateway, which goes to my Lambda and runs this:

import json
from truffleHog import truffleHog

def handler(event, context):
    try:
        print("EVENT:")
        print(event)
        print("CONTEXT:")
        print(context)
        print("BODY:")
        print(event['body'])
        githubData = json.loads(event['body'])
        print("COMMIT:")
        print(githubData['before'])
        print("URL:")
        print(githubData['repository']['url'])
        print("truffleHog:")
        print(truffleHog.find_strings(githubData['repository']['url'], do_regex=True, do_entropy=True, max_depth=100000, since_commit=githubData['before'], printJson=True, surpress_output=False))
        #print(truffleHog.find_strings(githubData['repository']['url'], do_regex=True, do_entropy=True, max_depth=100000))
        response = {
            "statusCode": 200,
            "body": "test",
        }
        return(response)
    except Exception as error:
        traceback.print_exc()

I'll keep trying to learn how Layers work to see if I can figure it out myself, but only started looking at them for the first time this morning. Could be something obvious that I've missed but just seems strange that it does work without the layer.

[mhart]

I get:

{
  "errorType": "NameError",
  "errorMessage": "name 'traceback' is not defined",
  "stackTrace": [
    "  File \"/var/task/handler.py\", line 26, in handler\n    traceback.print_exc()\n"
  ]
}

[mhart]

If I import traceback it seems to import fine

[LewisLebentz]

Hmm ok, must be something wrong with my setup then! I'll try deleting and redeploying the Lambda and dependencies when I'm back on my work laptop tomorrow.

Apologies! Thanks for your help.

[mhart]

No probs – to clarify, I got it running fine under docker-lambda – which is a pretty good proxy of the live Lambda environment. But there's still a chance that there's some permission or something on Lambda that might get in the way. I wouldn't have expected it to crash at the import stage though.

[LewisLebentz]

Thanks, just thought I'd confirm - it does work with Python.

Redeployed a fresh version of my Lambda and it worked!