search

k2-fsa / k2

FSA/FST algorithms, differentiable, with PyTorch compatibility.
https://k2-fsa.github.io/k2
Apache License 2.0
1.08k stars 211 forks source link

fail k2.compose(H, LG, inner_labels="tokens") #1285

Open liveroomand opened 2 months ago

liveroomand commented 2 months ago

when i run local/test.py, fail in line 136 --->HLG = k2.compose(H, LG, inner_labels="tokens")

error:

/var/www/k2/csrc/array_ops.cu:279:void k2::RowSplitsToRowIds(const k2::Array1&, k2::Array1*) Check failed: num_elems == row_splits[num_rows] (275402852 vs. 30203481)

[ Stack-Trace: ] /opt/conda/lib/python3.10/site-packages/k2/lib64/libk2_log.so(k2::internal::GetStackTrace()+0x34) [0x7f0e202a19b4] /opt/conda/lib/python3.10/site-packages/k2/lib64/libk2context.so(k2::internal::Logger::~Logger()+0x2a) [0x7f0e2083cd4a] /opt/conda/lib/python3.10/site-packages/k2/lib64/libk2context.so(k2::RowSplitsToRowIds(k2::Array1 const&, k2::Array1)+0x202) [0x7f0e20856952] /opt/conda/lib/python3.10/site-packages/k2/lib64/libk2context.so(k2::IndexAxis0(k2::RaggedShape&, k2::Array1 const&, k2::Array1)+0x405) [0x7f0e20a55fc5] /opt/conda/lib/python3.10/site-packages/k2/lib64/libk2context.so(k2::Index(k2::RaggedShape&, int, k2::Array1 const&, k2::Array1*)+0x36b) [0x7f0e20a56efb] /opt/conda/lib/python3.10/site-packages/_k2.cpython-310-x86_64-linux-gnu.so(+0x1b2502) [0x7f0e263ad502] /opt/conda/lib/python3.10/site-packages/_k2.cpython-310-x86_64-linux-gnu.so(+0x17b6de) [0x7f0e263766de] /opt/conda/lib/python3.10/site-packages/_k2.cpython-310-x86_64-linux-gnu.so(+0x158284) [0x7f0e26353284] /opt/conda/lib/python3.10/site-packages/_k2.cpython-310-x86_64-linux-gnu.so(+0x3dda7) [0x7f0e26238da7] python3() [0x4fd907] python3(_PyObject_MakeTpCall+0x25b) [0x4f705b] python3() [0x5098bf] python3(_PyEval_EvalFrameDefault+0x13b3) [0x4ef0e3] python3(_PyFunction_Vectorcall+0x6f) [0x4fdd4f] python3(_PyEval_EvalFrameDefault+0x4b26) [0x4f2856] python3(_PyFunction_Vectorcall+0x6f) [0x4fdd4f] python3(_PyEval_EvalFrameDefault+0x13b3) [0x4ef0e3] python3(_PyFunction_Vectorcall+0x6f) [0x4fdd4f] python3(_PyEval_EvalFrameDefault+0x13b3) [0x4ef0e3] python3(_PyFunction_Vectorcall+0x6f) [0x4fdd4f] python3(_PyEval_EvalFrameDefault+0x31f) [0x4ee04f] python3(_PyFunction_Vectorcall+0x6f) [0x4fdd4f] python3(_PyEval_EvalFrameDefault+0x31f) [0x4ee04f] python3() [0x5951c2] python3(PyEval_EvalCode+0x87) [0x595107] python3() [0x5c5ef7] python3() [0x5c1030] python3() [0x459781] python3(_PyRun_SimpleFileObject+0x19f) [0x5bb5bf] python3(_PyRun_AnyFileObject+0x43) [0x5bb323] python3(Py_RunMain+0x38d) [0x5b80dd] python3(Py_BytesMain+0x39) [0x5883f9] /usr/lib/x86_64-linux-gnu/libc.so.6(+0x29d90) [0x7f0ed94cdd90] /usr/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0x80) [0x7f0ed94cde40] python3() [0x5882ae]

Traceback (most recent call last): File "/home/liweiyong/project/icefall/egs/commonvoice/ASR/./local/compile_hlg.py", line 168, in logging.basicConfig(format=formatter, level=logging.INFO) File "/home/liweiyong/project/icefall/egs/commonvoice/ASR/./local/compile_hlg.py", line 158, in main logging.info(f"Processing {lang_dir}") File "/home/liweiyong/project/icefall/egs/commonvoice/ASR/./local/compile_hlg.py", line 136, in compile_HLG

also change other places in icefall that are using

File "/opt/conda/lib/python3.10/site-packages/k2/fsa_algo.py", line 486, in compose ans = intersect(a_fsa_inv, File "/opt/conda/lib/python3.10/site-packages/k2/fsa_algo.py", line 407, in intersect out_fsa = k2.utils.fsa_from_binary_function_tensor(a_fsa, b_fsa, File "/opt/conda/lib/python3.10/site-packages/k2/utils.py", line 629, in fsa_from_binary_functiontensor value, = b_value.index(b_arc_map, RuntimeError: Some bad things happened. Please read the above error messages and stack trace. If you are using Python, the following command may be helpful:

  gdb --args python /path/to/your/code.py

(You can use `gdb` to debug the code. Please consider compiling
a debug version of k2.).

If you are unable to fix it, please open an issue at:

  https://github.com/k2-fsa/k2/issues/new
pkufool commented 1 month ago

Can you clarify the size of your H and LG? I suspect it is an overflow of int32.

ALIVE321 commented 1 month ago

Similar to #1010. Any way to solve this?

liveroomand commented 1 month ago

2024-05-10 03:23:26,208 INFO [compile_hlg.py:133] H shape before Composing HLG: (501, None) 2024-05-10 03:23:26,208 INFO [compile_hlg.py:134] LG shape before Composing HLG: (11745505, None)

liveroomand commented 1 month ago

Similar to #1010. Any way to solve this?

That's true, Smaller LG Compose succeeded （H.shape: 501, LG.shape: 11745505）, the larger LG Compose fail （H.shape: 501, LG.shape: 51395348）. the larger LG there is no other good way to Compose. how much is the upper limit of Compose

danpovey commented 1 month ago

If this is a graph-building exercise of some kind it may be better to use the kaldifst project https://k2-fsa.github.io/kaldifst/ to do the graph-building, then if needed it can be converted to the k2 format.

ALIVE321 commented 1 month ago

If this is a graph-building exercise of some kind it may be better to use the kaldifst project https://k2-fsa.github.io/kaldifst/ to do the graph-building, then if needed it can be converted to the k2 format.

I've tried KaldiFST following the PR here, but the result just did not change.

pkufool commented 1 month ago

What do you mean "just did not change"?

ALIVE321 commented 1 month ago

What do you mean "just did not change"?

I got the same error as above, after the optional OpenFST procedures.

pkufool commented 1 month ago

You mean you can not compile HLG even with openfst?

By using openfst, you should convert H and LG from k2 to openfst, then compile HLG using openfst, convert the HLG back to k2 at last.

csukuangfj commented 1 month ago

What do you mean "just did not change"?

I got the same error as above, after the optional OpenFST procedures.

Could you show the error logs after switching to kaldifst?

ALIVE321 commented 1 month ago

What do you mean "just did not change"?

I got the same error as above, after the optional OpenFST procedures.

Could you show the error logs after switching to kaldifst?

Sorry that I didn't save the logs. I followed the LibriSpeech script in egs/librispeech/ASR/local/compile_hlg_using_openfst.py.

It started running slowly with openfst for several hours, and the same error as mentioned in this issue happened again when composing HLG at Line-156.

My lm.arpa has 80 million grams with order 1 to 3.

liveroomand commented 1 month ago

openfst

k2 to openfst use "from kaldifst.utils import k2_to_openfst ---> H_fst = k2_to_openfst(H)". compose use "HLG_fst = kaldifst.compose(H_fst, LG_fst)"， it success. how to convert openfst to k2

ALIVE321 commented 1 month ago

openfst

k2 to openfst use "from kaldifst.utils import k2_to_openfst ---> H_fst = k2_to_openfst(H)". compose use "HLG_fst = kaldifst.compose(H_fst, LG_fst)"， it success. how to convert openfst to k2

Check files of https://github.com/k2-fsa/icefall/pull/606. The conversion between k2 and fst is successful, but I guess the problem here is that k2.compose is still used after converting fst to k2.