python3.9 support

[clement-inventia]

Hi, First of all really appreciate your work and this repo, it's great to have such a resource to use in AWS! I was wondering if python3.9 support was under work? I am trying to migrate from 3.8 to 3.9 and build a lambda layer myself but I get stuck on cffi backend not being imported correctly. Happy to give more context and/or help with a potential PR!

[kalekseev]

Can you build and test layer in this branch https://github.com/kotify/cloud-print-utils/tree/sam-images with RUNTIME=python3.9 make build/weasyprint-layer-python3.9.zip

[clement-inventia]

Sorry for the delay, it took me a while to be able to test it. But it works! Thank you for the support. Something that took me a while to realise was that the docker platform was wrong when building from my M1 mac. After seeing your code, especially https://github.com/kotify/cloud-print-utils/blob/sam-images/Makefile#L3, I managed to fix it.

Thanks again for the support and the quick answer!

[elrehm]

Hello, I was able to use this branch to successfully build a weasyprint python3.9 layer for AWS linux/x86_64. I had to make two changes to do so:

1. Edited Makefile line 3 to docker run --rm --platform linux/x86_64
2. in weasyprint/layer_builder.sh, it was not finding the RUNTIME variable, so it was putting files in /opt/python/lib/site-packages instead of /opt/python/lib/python3.9/site-packages as expected. I hardcoded the paths to included 'python3.9' on lines 35 and 36 for my use.

(without those steps the layer couldn't find the cffi_backend)

[kalekseev]

@elrehm

Edited Makefile line 3 to docker run --rm --platform linux/x86_64

linux/amd64 is certainly supported https://docs.docker.com/build/building/multi-platform/ try to update your docker

in weasyprint/layer_builder.sh, it was not finding the RUNTIME variable, so it was putting files in /opt/python/lib/site-packages instead of /opt/python/lib/python3.9/site-packages as expected. I hardcoded the paths to included 'python3.9' on lines 35 and 36 for my use.

In my comment https://github.com/kotify/cloud-print-utils/issues/10#issuecomment-1367774956 full command provided including RUNTIME env variable RUNTIME=python3.9 make build/weasyprint-layer-python3.9.zip

[elrehm]

yes, to be more clear: I didn't get any errors building the zip file with linux/amd64, but needed linux/x86_64 for my use case. I did include RUNTIME=python3.9 in my make command as suggested, but it was empty in weasyprint/layer_builder.sh.

[StrongereCommerce]

I tested the layer in this branch https://github.com/kotify/cloud-print-utils/tree/sam-images with RUNTIME=python3.9 make build/weasyprint-layer-python3.9.zip I then pushed the layer my AWS account and added it to my lambda python 3.9 function.

But when I tried testing the function, I got:

{
  "errorMessage": "Unable to import module 'lambda_function': No module named 'weasyprint'",
  "errorType": "Runtime.ImportModuleError",
  "requestId": "6179e18e-e5ba-4329-b2b3-a7308dc9ab68",
  "stackTrace": []
}

Here is my function code:

import json

def lambda_handler(event, context):
    try:
        from weasyprint import HTML, CSS
        from weasyprint.text.fonts import FontConfiguration

        return {
            'statusCode': 200,
            'body': json.dumps('All modules are correctly installed.')
        }
    except Exception as e:
        return {
            'statusCode': 400,
            'body': json.dumps('Some modules are not correctly installed: ' + str(e))
        }

[StrongereCommerce]

@kalekseev Also, when I tried to test it locally (pythonmake test.weasyprint), I got the same error:

jayf@Jays-iMac-2 cloud-print-utils % make test.weasyprint
docker run --rm --platform linux/amd64 \
            -e GDK_PIXBUF_MODULE_FILE="/opt/lib/loaders.cache" \
            -e FONTCONFIG_PATH="/opt/fonts" \
            -e XDG_DATA_DIRS="/opt/lib" \
            -v `pwd`/weasyprint:/var/task \
            -v `pwd`/build/opt:/opt \
            lambci/lambda:python3.8 \
            lambda_function.lambda_handler \
            '{"url": "https://weasyprint.org/samples/report/report.html", "filename": "report.pdf", "return": "base64"}' \
            | tail -1 | jq .body | tr -d '"' | base64 -d > report.pdf
/bin/sh: jq: command not found
Unable to find image 'lambci/lambda:python3.8' locally
python3.8: Pulling from lambci/lambda
b8f7c23f9c29: Pull complete 
491e0bc29828: Pull complete 
0a7671393f66: Pull complete 
Digest: sha256:be943e04cfeda15e0ea141d84a914b12f500194a694e809bb3cd0d0dd187aa56
Status: Downloaded newer image for lambci/lambda:python3.8
START RequestId: 0a68fd4a-3f3a-1ad4-b989-f0a5649d0a6e Version: $LATEST
[ERROR] Runtime.ImportModuleError: Unable to import module 'lambda_function': No module named 'weasyprint'
END RequestId: 0a68fd4a-3f3a-1ad4-b989-f0a5649d0a6e
REPORT RequestId: 0a68fd4a-3f3a-1ad4-b989-f0a5649d0a6e  Init Duration: 1302.31 ms       Duration: 67.11 ms       Billed Duration: 68 ms  Memory Size: 1536 MB    Max Memory Used: 79 MB
Check ./report.pdf, eg.: xdg-open report.pdf

[jkmcclinton]

@StrongereCommerce I was able to get it working by moving all the Python libraries from weasyprint-layer-python3.9.zip/python/lib/site-packages to the python folder and was able to generate a pdf with it from a Lambda. In my experience working with Python Lambda layers, the packages have generally been there.