zkonduit / ezkl

ezkl is an engine for doing inference for deep learning models and other computational graphs in a zk-snark (ZKML). Use it from Python, Javascript, or the command line.
https://docs.ezkl.xyz/
947 stars 142 forks source link

ezkl cli gen-settings: [graph] a node is missing required params: padding #840

Closed pdibitonto closed 1 month ago

pdibitonto commented 2 months ago

I was testing ezkl with a publicly available model YOLOv8s-worldv2 but I'm receiving an error when I fire the gen-settings command, the error is the following: [graph] a node is missing required params: padding

Steps to reproduce the bug

To reproduce the bug:

  1. Download the YOLOv8s-worldv2 model here

  2. Export the model to onnx with the following python script:

    from ultralytics import YOLO
    model = YOLO("yolov8s-worldv2.pt")
    model.export(format = "onnx")
  3. In the same directory, try the gen-settings cli command as follow:

    ezkl gen-settings --model yolov8s-worldv2.onnx --input-visibility hashed --output-visibility public

Device and Operating System

OS: Ubuntu 22.04.4 Version: 22.04.4 LTS Processor: Intel® Core™ i7-10750H Memory: 32 GB

Additional Information

  1. Python version: 3.8.16+
  2. Required python module for onnx export: ultralytics
pdibitonto commented 2 months ago

Dunno if those details are useful, but I've printed the model (already exported to onnx) graph padding values and they are presents for each node, I've used this script:

import onnx

# Load the ONNX model
model = onnx.load("yolov8s-worldv2.onnx")

# Access the graph
graph = model.graph

for node in graph.node:
    if node.op_type == 'Conv':  # Look for Convolution nodes
        print(f"Node name: {node.name}")
        for attr in node.attribute:
            if attr.name == 'pads':  # 'pads' attribute contains padding info
                print(f"Padding: {attr.ints}")

Let me know if others infos are needed

lnm98 commented 1 month ago

Here's a script that creates a tiny onnx file which reproduces the problem:

import tensorflow as tf
import tf2onnx
import onnx

model = tf.keras.Sequential()
model.add(tf.keras.layers.MaxPooling2D(
    pool_size=(4, 4), strides=None, padding="same",
    )
)
model.output_names=['output']
input_signature = [tf.TensorSpec([20, 20, 1, 1], tf.float32, name='x')]
onnx_model, _ = tf2onnx.convert.from_keras(model, input_signature, opset=13)

onnx.save(onnx_model, "samepadding.onnx")

The bug seems to be occurring because the onnx parser doesn't handle pooling ops with size-preserving padding: https://github.com/zkonduit/ezkl/blob/64fbc8a1c9cca0aaee006208ab0484e136c71a73/src/graph/utilities.rs#L99-L106

alexander-camuto commented 1 month ago

@pdibitonto @lnm98 patching this asap :)