Cryptography issues

[dszakallas]

Hi, I've been trying to use a recent version of cryptography with graalpython native-image, but I am facing the following problems.

1. I've tried to install cryptography into a plain python virtualenv. This way I get the following error when running the application:

   Exception in thread "main" ImportError: cannot import name 'check_ansix923_padding' from 'cryptography.hazmat.bindings._rust' (None)
   at org.graalvm.polyglot.Context.eval(Context.java:425)
   at Test.main(Test.java:29)

   To reproduce, run

   docker buildx build . --target plain -t crypto-plain --progress plain
   docker run crypto-plain 

2. I've also tried installing graalpython environment first, and install cryptography into that. In this case pip would like to build it from source, but fails with an error:

   error: unknown interpreter: GraalVM

   docker buildx build . --target graal -t crypto-graal --progress plain
   docker run crypto-graal

   Here's the Dockerfile to reproduce the two attempts:

# syntax=docker/dockerfile:1.4

#########################################################
FROM scratch as wrapper
COPY <<"EOF" Test.java

import org.graalvm.polyglot.*;
import org.graalvm.polyglot.proxy.*;
import org.graalvm.polyglot.Context.Builder;

public class Test {
  public static String PYTHON = "python";

  public static String PROGRAM = """
    from cryptography.fernet import Fernet
    key = Fernet.generate_key()
    f = Fernet(key)
    token = f.encrypt(b"A really secret message. Not for prying eyes.")
    print(f.decrypt(token))
    """;

  public static void main(String[] args) {
    Builder ctxBuilder = Context
        .newBuilder(PYTHON)
        .allowAllAccess(true)
        .option("python.ForceImportSite", "true");

    String pythonPath = System.getenv("PYTHONPATH");
    if (pythonPath != null) {
      ctxBuilder.option("python.PythonPath", pythonPath);
    }

    try (Context context = ctxBuilder.build();) {
      context.eval(PYTHON, PROGRAM);
    }
  }
}
EOF

#########################################################
FROM python:3.8-slim as base

ARG APP_PATH=/app

WORKDIR $APP_PATH

ENV LANGUAGE=en_US.UTF-8
ENV LANG=en_US.UTF-8
ENV LC_ALL=en_US.UTF-8
ENV LC_CTYPE=en_US.UTF-8
ENV LC_MESSAGES=en_US.UTF-8

RUN set -ex \
    && apt-get update -yqq \
    && mkdir -pv /usr/share/man/man1 \
    && mkdir -pv /usr/share/man/man7 \
    && apt-get install -yqq --no-install-recommends \
        locales \
    && sed -i 's/^# en_US.UTF-8 UTF-8$/en_US.UTF-8 UTF-8/g' /etc/locale.gen \
    && locale-gen \
    && update-locale \
    && apt-get install -yqq --no-install-recommends \
      build-essential \
      curl \
      libssl-dev \
      libffi-dev \
      libz-dev \
    && pip install virtualenv \
    && curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y \
    && apt-get autoremove -yqq --purge \
    && apt-get clean \
    && rm -rf \
        /var/lib/apt/lists/* \
        /tmp/* \
        /var/tmp/* \
        /usr/share/man \
        /usr/share/doc \
        /usr/share/doc-base

ENV PATH=$HOME/.cargo/bin:$PATH

#########################################################
FROM base as graal

RUN cd /opt/ && curl -fsSL https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-22.1.0/graalvm-ce-java17-linux-amd64-22.1.0.tar.gz | tar -xz

ENV PATH=/opt/graalvm-ce-java17-22.1.0/bin:$PATH \
    JAVA_HOME=/opt/graalvm-ce-java17-22.1.0 \
    GRAALVM_HOME=/opt/graalvm-ce-java17-22.1.0

RUN gu install python native-image \
    && graalpython -m venv $APP_PATH 

RUN . $APP_PATH/bin/activate \
    && . $HOME/.cargo/env \
    && pip install --no-cache "cryptography==37.0.2"

COPY --from=wrapper Test.java Test.java

RUN javac Test.java \
    && native-image --language:python Test test \
        -Dorg.graalvm.launcher.relative.python.home=$GRAALVM_HOME/languages/python \
        -Dorg.graalvm.launcher.relative.llvm.home=$GRAALVM_HOME/languages/llvm \
    && chmod +x test && mv test $APP_PATH/bin

ENV PYTHONPATH=$APP_PATH/lib/python3.8/site-packages

ENV PATH=$APP_PATH/bin:$PATH

ENTRYPOINT ["test"]

#########################################################
FROM base as plain

RUN virtualenv -p python3.8 $APP_PATH

RUN . $APP_PATH/bin/activate \
    && pip install --no-cache "cryptography==37.0.2"

RUN cd /opt/ && curl -fsSL https://github.com/graalvm/graalvm-ce-builds/releases/download/vm-22.1.0/graalvm-ce-java17-linux-amd64-22.1.0.tar.gz | tar -xz

ENV PATH=/opt/graalvm-ce-java17-22.1.0/bin:$PATH \
    JAVA_HOME=/opt/graalvm-ce-java17-22.1.0 \
    GRAALVM_HOME=/opt/graalvm-ce-java17-22.1.0

RUN gu install python native-image \
    && graalpython -m venv $APP_PATH 

COPY --from=wrapper Test.java Test.java

RUN javac Test.java \
    && native-image --language:python Test test \
        -Dorg.graalvm.launcher.relative.python.home=$GRAALVM_HOME/languages/python \
        -Dorg.graalvm.launcher.relative.llvm.home=$GRAALVM_HOME/languages/llvm \
    && chmod +x test && mv test $APP_PATH/bin

ENV PYTHONPATH=$APP_PATH/lib/python3.8/site-packages

ENV PATH=$APP_PATH/bin:$PATH

ENTRYPOINT ["test"]

[msimacek]

Hi @dszakallas, thank you for the report. The rust bindings to cryptography currently don't work. I was able to hack together some scripts that make the rust code build with our toolchain and I've hit an issue on the side of Sulong, our LLVM runtime, that it currently cannot pass structs by-value to native code, which is needed to call into the rust standard library. The issue is being worked on by the Sulong team. In the meantime, you can use cryptography 3.4.7, which I believe was the last version that works without the rust bindings.

[msimacek]

We now have patches for multiple recent versions of cryptography that should work (we've had them for some time, but I forgot to update this issue).