bug: `on-demand` `Memory` circuit fetching error

[Nashtare]

It seems there's some inconsistency in circuit fetching when running zero-bin with --load-strategy on-demand. In particular it seems to always affect the Memory circuit.

The logs seem to highlight the expected configuration is running:

INFO p_gen: zero_bin_common::prover_state:  using on demand circuit ProverStateManager { circuit_config:
    CircuitConfig { circuits: [16..26, 8..23, 12..25, 14..23, 9..20, 12..22, 17..28] }

where the Memory range is 17..28.

Several witnesses failed with following errors:

 Error: Fatal operation error: Attempting to load circuit: Memory at size: 21
 Error: Fatal operation error: Attempting to load circuit: Memory at size: 19
 Error: Fatal operation error: Attempting to load circuit: Memory at size: 19
 ...

When inspecting zero_bin/circuits it seems the circuits have been properly written to disk:

Obtained on zk_evm@feat/cancun, commit ddca6be9a40a7d0492312ecaf9d966e2124dbc91.

Side note:

• it seems the verifier circuit writing to disk is inconsistent, cf image above (purged state prior). This may be due to the latest refactoring on package version fetching? (cc @0xaatif)

[0xaatif]

It's plausible! could you paste instructions on how to repro?

[Nashtare]

Sure. I've replaced the circuit sizes in prove_stdio.sh with

        export ARITHMETIC_CIRCUIT_SIZE="16..26"
        export BYTE_PACKING_CIRCUIT_SIZE="8..23"
        export CPU_CIRCUIT_SIZE="12..25"
        export KECCAK_CIRCUIT_SIZE="14..23"
        export KECCAK_SPONGE_CIRCUIT_SIZE="9..20"
        export LOGIC_CIRCUIT_SIZE="12..22"
        export MEMORY_CIRCUIT_SIZE="17..28"

and on the branch / commit mentioned, running ./prove_stdio.sh witness.json Note that you may need to set -n 1 (num workers) or use a powerful machine to handle the RAM consumption.

[Nashtare]

This one has not been observed / reproduced since proper stabilization of zero-bin post migration into zk_evm and correct handling around circuit persistence logic, hence will close for now and reopen if it ever occurs again.