sonos / tract

Tiny, no-nonsense, self-contained, Tensorflow and ONNX inference
Other
2.23k stars 214 forks source link

Failed analyse for node #529 "/Add" Add #1237

Closed bayedieng closed 1 year ago

bayedieng commented 1 year ago

I'm getting this error while I run my model:

Error: ModelError(Failed analyse for node #529 "/Add" Add

Caused by:
    0: Infering facts
    1: Applying rule WithRule { inputs[1].shape }
    2: Matching 1,192,831 and 1,192,audio_data with numpy/onnx broadcast rules
    3: Invalid shape (broadcasting): Sym(audio_data) is not compatible with Some(Val(831)).)

Interestingly enough it works when I run it unoptimized, however I get the following:

thread 'main' panicked at 'index out of bounds: the len is 2 but the index is 2', /Users/bdieng/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tract-onnx-0.20.20/src/ops/array/pad.rs:105:48
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

Here is my model code:

    match model_inputs {
        ModelInputs::HubertModel(input) => {
            let hubert = tract_onnx::onnx()
                .model_for_path(model_path)?
                .into_optimized()?
                .into_runnable()?;

            let input_tensor: Tensor = input.into();
            let output = hubert.run(tvec![input_tensor.into()])?;
            let output_tensor = &output[0];
            let shape = output_tensor.shape();
            Ok(ModelOutputs::HubertModel(
                output_tensor
                    .to_array_view::<f32>()?
                    .into_shape((shape[0], shape[1], shape[2]))
                    .unwrap()
                    .to_owned(),
            ))
        }
        ModelInputs::VoiceConversionModel(vc_input) => {
            let vc_model = tract_onnx::onnx()
                .model_for_path(model_path)?
                .into_optimized()?
                .into_runnable()?;
            println!("{:?}", &vc_input.c.shape());

            let c_tensor: Tensor = vc_input.c.into();
            let f0_tensor: Tensor = vc_input.f0.into();
            let uv_tensor: Tensor = vc_input.uv.into();
            let output =
                vc_model.run(tvec![c_tensor.into(), f0_tensor.into(), uv_tensor.into()])?;

            let output_tensor = &output[0];
            Ok(ModelOutputs::VoiceConversionModel(
                output_tensor.as_slice::<f32>()?.to_vec(),
            ))
        }

This model uses so-vits-svc-fork which is a voice conversion model and i made sure when exporting the pytorch to onnx to make all the axes dynamic where it matters. Despite that, as stated, even when inference is done I run into a bounds error through ndarray. I have already tried running the model through onnxruntime and have gotten it to work. Would like to note that the hubert model runs, it's the voice conversion that returns the error.

bayedieng commented 1 year ago

After some optimazation and some analysis using the tract cli I have determined that the model employs a LazyConv operator that tract does not seem support.