Open uni-sagar-raikar opened 1 year ago
Is it reproducible? Could you print the input tensor shape (N, T, C) to the model?
Hi, Yes it is reproducible. We could reproduce it with 2 audio files so far, of 1599 and 1600 sample sizes. Definitely small.
(1 x 82). Kernel size: (3 x 3). Kernel size can't be greater than actual input size
(2 x 39). Kernel size: (3 x 3). Kernel size can't be greater than actual input size
If the sampling rate is 16 kHz, then 1600 samples correspond to only 0.1 second or about 10 feature frames, which is far too small.
Does it crash if you use a longer audio file?
Thanks for the quick response. No, it works okay with longer files. Not sure about the exact threshold, but as a quick fix, we filter out everything below 300ms on the client side.
I traced back the error with debugger up to OfflineConformerTransducerModel::RunEncoder
where a dynamic function call is performed, calling "forward" on zipformer.py.
The input has to have at least 9 feature frames to avoid crashes.
Could you print the input tensor shape?
We suggest that the input tensor has to have 23 feature frames.
If you only have 9 feature frames, then after the first down-sampling layer only 1 feature frame is left. It is hard to decode something from only a single frame.
We suggest that the input tensor has to have 23 feature frames.
Thank you for the comment and the suggestion.
I would like to take some more steps towards avoiding server crashes.
(1) Is the minimal size requirement model/architecture specific? -> if yes, is there a way to know it - by querying the model?
(2) Would a more aggressive padding on the sample (or feature) domain solve the issue?
Again, thank you for your helpful comments.
(1) Is the minimal size requirement model/architecture specific?
Yes, it is hard-coded in the model since all zipformer models from icefall have such a constraint.
(2) Would a more aggressive padding on the sample (or feature) domain solve the issue?
Yes, I think it would fix the issue by padding the input wave.
Hi,
We are facing the following issue with offline-websocket-server when used with zipformer1 model.
`terminate called after throwing an instance of 'std::runtime_error' what(): The following operation failed in the TorchScript interpreter. Traceback of TorchScript, serialized code (most recent call last): File "code/torch/zipformer.py", line 23, in forward _0 = torch.icefall.utils.make_pad_mask encoder_embed = self.encoder_embed x0 = (encoder_embed).forward(x, )