How to run on arm64 architecture

[Charitra1]

I changed the python runtime to 3.8 (from 3.6) in config/example.json which was shown exactly (3.8) in aws lambda function in the management console. In addition, I have also added one parameter architectures: ["arm64"] under aws in the same file (config/example.json). However, the architecture did not change, it was still showing x86_64. My question is, what parameter should i need to add to change the architecture to arm64??

[mcopik]

@Charitra1 Dear Charitra, we currently do not support configuring CPU architecture for the functions. That's why the additional setting in the JSON file has been simply ignored. Did you find anything in our documentation that indicated such support?

It should be relatively easy to add this option. I assume that you want to work with Graviton functions on AWS Lambda?

[Charitra1]

@mcopik Dear Marcin, Thank you for your response I have a question I have tried manually changing the architecture to arm64 (at management console) and then invoking the benchmark again. By doing so, can we consider the latest output has been generated using arm64? OR, Can you please tell what are the files that i need to work on to change the arch. to arm64?

[mcopik]

We should extend our function configuration to support the new "architecture" type, and use it when deploying functions to AWS Lambda.

[rahulsurwade08]

Hey @mcopik Can you elaborate the issue for better understanding? Thanks!

[mcopik]

@rahulsurwade08 I realized the issue is a bit more complex because of the binary dependencies of our functions - see an article on the AWS website. For example, the benchmark dynamic-html would likely work out of the box on an ARM function on AWS Lambda since it contains only Python code and uses a small Python library. On the other hand, many other benchmarks use Python packages with an embedded C library - for these, we need to install aarch64 versions from pip.

It's more complex, so another issue might be a good starting point - but it's still an entertaining and interesting one :-)

Steps to solve

• [ ] We already have the configuration for creating functions with different architectures. We do not use it yet to create the Lambda function.
• [ ] Install pip dependencies to arm64 platform, e.g., pip install --platform manylinux_2_17_aarch64 --only-binary=:all: --target .deps numpy==1.24.2.
• [ ] Expand benchmark configuration to support different dependencies and their versions for different architectures.
• [ ] Establish which benchmarks require compiling dependencies from source with the help of pip, and adjust Docker images and benchmark configuration to support his.
• [ ] We are missing the documentation for function runtime.

[saurabhmj11]

In order to run on arm64 architecture, you need to ensure that your AWS Lambda function's deployment package contains compatible binaries and dependencies for the arm64 architecture.

To do this, you can follow these steps:

Update the runtime field in your config/example.json file to specify the Python version and architecture that you want to use. For example: json Copy code "runtime": "python3.8", "architectures": ["arm64"] Build your deployment package on an arm64 machine or in a compatible Docker container to ensure that the binaries and dependencies are compatible with the arm64 architecture.

Upload the deployment package to AWS Lambda and create a new function with the updated runtime and architectures fields in your config/example.json file.

Once your function is deployed, you can verify that it is running on the arm64 architecture by checking the architecture field in the AWS Lambda management console or by using the AWS CLI command aws lambda get-function-configuration to retrieve the function configuration.

[mcopik]

@saurabhmj11 Yes. The complexity comes primarily from building native dependencies into arm64.

[prajinkhadka]

HI @mcopik, I will be working on this issue.

Let me know if there are any other things I need to be aware of.

[mcopik]

@prajinkhadka Thanks for your interest! Please focus on the benchmarks that have binary dependencies, like thumbnailer that depends on Pillow. These will be the most difficult to make sure that we install the ARM version.

Some benchmarks, like image recognition, might not work due to the requirement for code package size as we would have to use a new and large PyTorch version, which are too large for Lambda's code package. We need to merge container support first to support this.

[MinhThieu145]

Hello @mcopik,

I have reviewed the problem and believe that implementing merge container support would be beneficial. I am considering using ECR by building Docker locally, then uploading it to ECR, and subsequently using it as the source for Lambda. However, I have one concern. To my knowledge, the code still checks for updates from Lambda (either directly or from S3). Unfortunately, this approach might not be feasible when using containers. I would really appreciate your feedback on this matter.

[mcopik]

@MinhThieu145 We have container support on a branch primarily focused on C++: https://github.com/spcl/serverless-benchmarks/tree/cpp_benchmarks

I will try to test it for Python/JS and merge soon.

[MinhThieu145]

Hello @mcopik,

Thanks for the info you shared. I have a few points of clarification regarding the ccp_branch, and I would appreciate your insights

1. Master Branch Process: Right now, the master branch has a process where files are organized into a folder, zipped, and then uploaded. There's also a _installdependencies function that uses Docker but only for Azure. Can you explain why this function is there? Since I mainly know about AWS, I'm wondering if this should also work for AWS and Google Cloud.

2. Docker and AWS: From what I know about AWS, you can't upload a Docker image to S3 and then use it with Lambda. But it looks like this is possible with Azure, which might be why Docker is only used for Azure here.

I looked at the cpp_benchmark repo and it seems like it only supports containers for Azure. If I'm missing something or if I've got something wrong, please correct me.

[prajinkhadka]

@rahulsurwade08 I realized the issue is a bit more complex because of the binary dependencies of our functions - see an article on the AWS website. For example, the benchmark dynamic-html would likely work out of the box on an ARM function on AWS Lambda since it contains only Python code and uses a small Python library. On the other hand, many other benchmarks use Python packages with an embedded C library - for these, we need to install aarch64 versions from pip.

It's more complex, so another issue might be a good starting point - but it's still an entertaining and interesting one :-)

Steps to solve

* [ ]  We already have [the configuration](https://github.com/spcl/serverless-benchmarks/blob/master/sebs/faas/function.py#L270) for creating functions with different architectures. We do not use it yet to [create the Lambda function](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/lambda/client/create_function.html).

* [ ]  Install pip dependencies to `arm64` platform, e.g.,  `pip install --platform manylinux_2_17_aarch64 --only-binary=:all: --target .deps numpy==1.24.2`.

* [ ]  Expand benchmark configuration to support different dependencies and their versions for different architectures.

* [ ]  Establish which benchmarks require compiling dependencies from source with the help of `pip`, and adjust Docker images and benchmark configuration to support his.

* [ ]  We are missing the documentation for function runtime.

hi @mcopik , I was looking into the codebase on how to solve this problem. Based on the current architecture, it seems the image used is "spcleth/serverless-benchmarks:build.aws.python.3.7" ( based on the python version ) the AMD 64-based image.

To build for the arm64 architecture, I was thinking of having another base docker image ( with arm64 as the base ) and using that to build the dependencies. So as far as I understand there are two approaches to this problem :

1. Use the same AMD based docker image and use flags in pip ( --platform )
2. Use another ARM based docker image.

Which one do you think will be a good idea? I am inclined towards the latter option because of packages having binary dependencies.

[mcopik]

@prajinkhadka I recommend starting with option nr 1 - we want to build a code package with ARM dependencies for ARM CPUs in the AWS cloud, but we need to run this on x86 machines.

[prajinkhadka]

@mcopik , I have sent a PR #192 for this issue. I did tests for all the benchmarks except for 411.image-recognition for python3.8 on lambda for both arm64 and x86_64 based architecture. Let me know what you think and if any changes re required.

For 411.image-recognition we may discuss how can we do it.