Groth16 Circom is slower than Rapidsnark in RISC0 wrapper

[thomaslavaur]

Issue type

Performance

OS platform and distribution

Ubuntu 22

Current behavior?

Machine: 13th Gen Intel(R) Core(TM) i9-13980HX I benchmark tachyon circom vendor and rapidsnark, and compare their performance over the RISC0 zkVM wrapper.

I obtained that: Succint proof (without wrapping): 74s Complete proof (with rapidsnark): 156s Complete proof (with Tachyon): 400s

Expected Behavior?

I want to figure out the reason.

Standalone code or description to reproduce the issue

I used their own script for rapidsnark and modified it to build the groth16 wrapper of RISC0 in docker:

# syntax=docker/dockerfile:1.7
FROM rust:1.74.0 AS dependencies

WORKDIR /src/

ARG PYTHON_VERSION=3.10.12

RUN apt update && \
    apt install -y --no-install-recommends \
    wget \
    ca-certificates && \
    rm -rf /var/lib/apt/lists/*
RUN wget https://github.com/bazelbuild/bazelisk/releases/latest/download/bazelisk-linux-amd64 && \
    chmod +x bazelisk-linux-amd64 && \
    mv bazelisk-linux-amd64 /usr/local/bin/bazel

ENV HOME /root
RUN apt update && \
    apt install -y --no-install-recommends \
    build-essential \
    curl \
    git \
    libbz2-dev \
    liblzma-dev \
    libncursesw5-dev \
    libreadline-dev \
    libsqlite3-dev \
    libssl-dev \
    make \
    python-is-python3 \
    tk-dev \
    xz-utils \
    zlib1g-dev && \
    rm -rf /var/lib/apt/lists/*
RUN curl https://pyenv.run | bash
RUN CONFIGURE_OPTS=--enable-shared $HOME/.pyenv/bin/pyenv install ${PYTHON_VERSION}
RUN mkdir -p $HOME/bin
ENV PATH="${HOME}/.pyenv/shims:${HOME}/.pyenv/bin:${HOME}/bin:${PATH}"

RUN apt update && \
    apt install -y --no-install-recommends \
    libgmp-dev \
    libomp-dev && \
    rm -rf /var/lib/apt/lists/*

ENV USE_BAZEL_VERSION=6.3.0
# Check why we need that
ENV CARGO_BAZEL_REPIN=true

WORKDIR /src/

# Build Tachyon
RUN git clone https://github.com/kroma-network/tachyon.git && \
  cd tachyon

WORKDIR /src/tachyon/vendors/circom/

RUN bazel build --config linux //:prover_main && \
    cp ./bazel-bin/prover_main /usr/local/sbin/rapidsnark

WORKDIR /src/

RUN apt -qq update && \
  apt install -y -q libgmp-dev libomp-dev zip wget python3-numpy curl python-is-python3 apt-transport-https build-essential clang cmake curl gnupg libsodium-dev m4 nasm nlohmann-json3-dev npm

# Build and install circom
RUN git clone https://github.com/iden3/circom.git && \
  cd circom && \
  git checkout e60c4ab8a0b55672f0f42fbc68a74203bdb6a700 && \
  cargo install --path circom

ENV CC=clang
ENV CXX=clang++

# Cache ahead of the larger build process
FROM dependencies AS builder

WORKDIR /src/
COPY groth16/risc0.circom ./groth16/risc0.circom
COPY groth16/stark_verify.circom ./groth16/stark_verify.circom

# Build the witness generation
RUN (cd groth16; circom --c stark_verify.circom) && \
  sed -i 's/g++/clang++/' groth16/stark_verify_cpp/Makefile && \
  sed -i 's/O3/O0/' groth16/stark_verify_cpp/Makefile && \
  (cd groth16/stark_verify_cpp; make)

# Download the proving key
RUN wget https://risc0-artifacts.s3.us-west-2.amazonaws.com/zkey/2024-05-17.1/stark_verify_final.zkey.gz -O groth16/stark_verify_final.zkey.gz && \
  (cd groth16; gzip -df stark_verify_final.zkey.gz)

# Create a final clean image with all the dependencies to perform stark->snark
FROM ubuntu:jammy-20231211.1@sha256:bbf3d1baa208b7649d1d0264ef7d522e1dc0deeeaaf6085bf8e4618867f03494 AS prover

RUN apt update -qq && \
  apt install -y libsodium23 nodejs npm libgmp-dev libomp-dev && \
  npm install -g snarkjs@0.7.3

COPY scripts/prover.sh /app/prover.sh
COPY --from=builder /usr/local/sbin/rapidsnark /usr/local/sbin/rapidsnark
COPY --from=builder /src/groth16/stark_verify_cpp/stark_verify /app/stark_verify
COPY --from=builder /src/groth16/stark_verify_cpp/stark_verify.dat /app/stark_verify.dat
COPY --from=builder /src/groth16/stark_verify_final.zkey /app/stark_verify_final.zkey

WORKDIR /app
RUN chmod +x prover.sh
RUN ulimit -s unlimited

ENTRYPOINT ["/app/prover.sh"]

Additional context

No response

[batzor]

Please run with maxopt like this:

RUN bazel build --config maxopt //:prover_main && \
    cp ./bazel-bin/prover_main /usr/local/sbin/rapidsnark

[thomaslavaur]

It's better but Tachyon is still slower. I changed my computer so here are the new performances:

Succint proof (without wrapping): 20s Complete proof (with rapidsnark): 50s Complete proof (with Tachyon): 104s

Can it be because the circuit is too large?

[chokobole]

@thomaslavaur Thanks for reporting the issue. We'll take a look at it and get back to you

[batzor]

bazel build --@kroma_network_tachyon//:has_openmp --config maxopt //:prover_main

@thomaslavaur Sorry you didn't enable multithreading as well. Please try like this.

In vendors/circom there is a README with guides on how to build and info on docker as well. I think you can refer to it when building your docker.

[thomaslavaur]

Thanks for your help. Now times are close to each other: Succint proof (without wrapping): 20s Complete proof (with rapidsnark): 52s Complete proof (with Tachyon): 50s

[batzor]

Closing the issue! Btw, we also have docker for CUDA build :)