_update_kv_and_cache func init cache report error

[JinmingChe]

Describe the bug Running wenet/bin/recognize.py reports the error: "ValueError: not enough values to unpack (expected 2, got 0)."

To Reproduce Run the wenet/bin/recognize.py script to infer the streaming efficient model (train_u2++_efficonformer_v1.yaml) with chunksize=18 and streaming=true.

I compared the old version, and the main difference in the code is shown below:

" old: if cache.size(0) > 0: key_cache, value_cache = torch.split( cache, cache.size(-1) // 2, dim=-1) k = torch.cat([key_cache, k], dim=2) v = torch.cat([value_cache, v], dim=2) new: key_cache, value_cache = cache if key_cache.size(0) > 0: k = torch.cat([key_cache, k], dim=seq_axis) if value_cache.size(0) > 0: v = torch.cat([value_cache, v], dim=seq_axis) "

When the default cache is passed to the _update_kv_and_cache function, the initial cache is torch.tensor([]), which cannot be split into key_cache and value_cache.

Expected behavior Traceback (most recent call last): File "/lib/python3.10/runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "/lib/python3.10/runpy.py", line 86, in _run_code exec(code, run_globals) File "/code-server/extensions/ms-python.debugpy-2024.6.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/main.py", line 39, in cli.main() File "/code-server/extensions/ms-python.debugpy-2024.6.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main run() File "/code-server/extensions/ms-python.debugpy-2024.6.0-linux-x64/bundled/libs/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file runpy.run_path(target, run_name="main") File "/code-server/extensions/ms-python.debugpy-2024.6.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path return _run_module_code(code, init_globals, run_name, File "/code-server/extensions/ms-python.debugpy-2024.6.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code _run_code(code, mod_globals, init_globals, File "/code-server/extensions/ms-python.debugpy-2024.6.0-linux-x64/bundled/libs/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code exec(code, run_globals) File "/wenet/bin/recognize_debug.py", line 321, in main() File "/wenet/bin/recognize_debug.py", line 292, in main results = model.decode( File /wenet/transformer/asr_model.py", line 295, in decode encoder_out, encoder_mask = self._forward_encoder( File "/wenet/transformer/asr_model.py", line 223, in _forward_encoder encoder_out, encoder_mask = self.encoder.forward_chunk_by_chunk( File "/wenet/efficient_conformer/encoder.py", line 548, in forward_chunk_by_chunk self.forward_chunk( File "/wenet/efficient_conformer/encoder.py", line 404, in forwardchunk xs, , new_att_cache, new_cnn_cache = layer( File "python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(*args, kwargs) File "python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, *kwargs) File "/wenet/transformer/encoder_layer.py", line 234, in forward x_att, new_att_cache = self.self_attn(x, x, x, mask, pos_emb, File "python3.10/site-packages/torch/nn/modules/module.py", line 1511, in _wrapped_call_impl return self._call_impl(args, kwargs) File "python3.10/site-packages/torch/nn/modules/module.py", line 1520, in _call_impl return forward_call(*args, **kwargs) File "wenet/transformer/attention.py", line 393, in forward k, v, new_cache = self._update_kv_and_cache(k, v, cache) File "wenet/transformer/attention.py", line 207, in _update_kv_and_cache key_cache, value_cache = cache ValueError: not enough values to unpack (expected 2, got 0)

[xingchensong]

The Cache API changed in this pull request (https://github.com/wenet-e2e/wenet/pull/2481). It is not modified (for export_onn_gpu.py) accordingly.

[xingchensong]

duplicate with https://github.com/wenet-e2e/wenet/issues/2653