Encountered errors with a multi-dataset zipformer streaming model

[jingzhaoou]

I tried to run sherpa-online with the following multi-dateset zipformer streaming model: https://huggingface.co/marcoyang/icefall-libri-giga-pruned-transducer-stateless7-streaming-2023-04-04 and ran into the following errors:

./build/bin/sherpa-online --use-gpu=false \
        --decoding-method=modified_beam_search \
        --num-active-paths=1 \
        --use-endpoint=false \
        --nn-model=./models/icefall-libri-giga-pruned-transducer-stateless7-streaming-2023-04-04/exp/cpu_jit.pt \
        --tokens=./models/icefall-libri-giga-pruned-transducer-stateless7-streaming-2023-04-04/data/lang_bpe_500/tokens.txt \
          ./models/icefall-asr-librispeech-pruned-transducer-stateless7-streaming-2022-12-29/test_wavs/1221-135766-0001.wav
[I] ../sherpa/csrc/parse-options.cc:495:int sherpa::ParseOptions::Read(int, const char* const*) 2023-06-13 00:08:16.276 ./build/bin/sherpa-online --use-gpu=false --decoding-metho
d=modified_beam_search --num-active-paths=1 --use-endpoint=false --nn-model=./models/icefall-libri-giga-pruned-transducer-stateless7-streaming-2023-04-04/exp/cpu_jit.pt --tokens=
./models/icefall-libri-giga-pruned-transducer-stateless7-streaming-2023-04-04/data/lang_bpe_500/tokens.txt ./models/icefall-asr-librispeech-pruned-transducer-stateless7-streaming
-2022-12-29/test_wavs/1221-135766-0001.wav

[I] ../sherpa/cpp_api/bin/online-recognizer.cc:142:int32_t main(int32_t, char**) 2023-06-13 00:08:16.276 decoding method: modified_beam_search
[I] ../sherpa/cpp_api/online-recognizer.cc:424:void sherpa::OnlineRecognizer::OnlineRecognizerImpl::WarmUp() 2023-06-13 00:08:17.318 WarmUp begins
terminate called after throwing an instance of 'c10::Error'
  what():  Expected at most 3 argument(s) for operator 'forward', but received 4 argument(s). Declaration: forward(__torch__.zipformer.Zipformer self, Tensor x, Tensor x_lens) ->
 ((Tensor, Tensor))
Exception raised from checkAndNormalizeInputs at ../aten/src/ATen/core/function_schema_inl.h:340 (most recent call first):

I then tried another zipformer model from https://k2-fsa.github.io/sherpa/cpp/pretrained_models/online_transducer.html#icefall-asr-librispeech-streaming-zipformer-2023-05-17, which works fine.

Is it true taht sherpa does not support some zipfromer streaming architectures? If yes, can someone shed some light on how we may support them? I am interested in helping out with some guidance from the community. Thanks a lot!

[trunglebka]

+1 here

[csukuangfj]

Is it true taht sherpa does not support some zipfromer streaming architectures? I

No, it is not true.

---

The exported model in https://huggingface.co/marcoyang/icefall-libri-giga-pruned-transducer-stateless7-streaming-2023-04-04 uses an old version of icefall. If you re-export it with the latest icefall, then everything should work fine.

Will update the exported model in https://huggingface.co/marcoyang/icefall-libri-giga-pruned-transducer-stateless7-streaming-2023-04-04

[jingzhaoou]

I verified that things work fine with sherpa after I re-exported the model with the latest icefall using the following command:

./pruned_transducer_stateless7_streaming_multi/export.py \
  --exp-dir ./icefall-libri-giga-pruned-transducer-stateless7-streaming-2023-04-04/exp \
  --bpe-model ./icefall-libri-giga-pruned-transducer-stateless7-streaming-2023-04-04/data/lang_bpe_500/bpe.model \
  --epoch 20 \
  --jit True \
  --avg 1 \
  --use-averaged-model False

[csukuangfj]

Is it true taht sherpa does not support some zipfromer streaming architectures? I

No, it is not true.

The exported model in https://huggingface.co/marcoyang/icefall-libri-giga-pruned-transducer-stateless7-streaming-2023-04-04 uses an old version of icefall. If you re-export it with the latest icefall, then everything should work fine.

Will update the exported model in https://huggingface.co/marcoyang/icefall-libri-giga-pruned-transducer-stateless7-streaming-2023-04-04

Just updated the model https://huggingface.co/marcoyang/icefall-libri-giga-pruned-transducer-stateless7-streaming-2023-04-04/discussions/1