alexcasalboni / aws-lambda-power-tuning

AWS Lambda Power Tuning is an open-source tool that can help you visualize and fine-tune the memory/power configuration of Lambda functions. It runs in your own AWS account - powered by AWS Step Functions - and it supports three optimization strategies: cost, speed, and balanced.
Apache License 2.0
5.27k stars 362 forks source link

Image in payload possibility #229

Closed dmatos2012 closed 5 months ago

dmatos2012 commented 5 months ago

Hello, Thanks for the amazing work. For my payload, I need to include an image, along with other fields. How would I go about the image? I saw that one option is encoding the image in base64, but this doesnt seem to nice as its such a large string, but even tho not sure if thats the correct approach.

I also saw the payloadS3 that references a .json in our s3 buckets, but that too would have the same problem. Currently I give it to an API via (e.g python) = file = open("myimage.jpeg", "rb").

Thank you for your time.

alexcasalboni commented 5 months ago

Hi @dmatos2012, thanks for reaching out 😄

I would normally recommend fetching images from S3 in your function's code, rather than expecting an inline/encoded image directly in the payload.

If you can accomplish that, then it will be very easy to power-tune.

In case that's not possible, you should use the payloadS3 parameter, otherwise Step Functions won't be able to pass the execution state around (if larger than 256KB).

Let me know if one of these options work for you 🚀

dmatos2012 commented 5 months ago

Fetching images from S3 in my function code would be the most sensible approach and makes the most sense(which i want to do either way). However, changing this with my current infra would take lots of changes so this is more on the long term, and I still want to analyze it.

When you mention payloadS3, I see in the docs the following : "payloadS3": "s3://your-bucket/your-object.json" Will this payload(in s3) contain the base64 encoded string like this?

your-object.json:

        "file": "base64stringbasedonimage.jpeg=",
        "analysis": {
            "name": "john",
            "age":10 ,
        }
    }

Or is there a possiblity to include a reference to an S3 object containing the image like :

your-object.json

"file": "s3://my-bucket/image.jpeg"
"analysis": {
       .....  
      }

Sorry if its asking too much, but want to know whether its possible to power-tune or just basically have to remove the image in the json if I want to test it.

Again thanks for your time, and looking forward to using it. :)

alexcasalboni commented 5 months ago

The first option :)

The json file will include the entire payload in JSON format, so the image would need to be encoded as a string.

Using something like s3://my-bucket/image.jpeg is the long term solution, as you mentioned. Then you won't even need payloadS3.

dmatos2012 commented 5 months ago

Thank you so much for the fast response, its pretty clear what I need to do. Awesome work btw!

alexcasalboni commented 5 months ago

Happy to help :)

I'm closing this issue for now, feel free to keep commenting in case you face any issues.