search

k2-fsa / icefall

https://k2-fsa.github.io/icefall/
Apache License 2.0
792 stars 267 forks source link

A question about the data preparation on AMI corpus #1579

Closed hiranoyu0830 closed 6 days ago

hiranoyu0830 commented 1 month ago

Hello, thank you for making such a great toolkit.

I am reproducing pruned_transducer_stateless7 on AMI corpus. When I applied GSS enhancer to training data (prepare.sh: stage 3, local/prepare_ami_gss.sh: stage 4), I encountered an assertion error below.

Original Traceback (most recent call last): File "/opt/conda/lib/python3.10/site-packages/torch/utils/data/_utils/worker.py", line 308, in _worker_loop data = fetcher.fetch(index) File "/opt/conda/lib/python3.10/site-packages/torch/utils/data/_utils/fetch.py", line 53, in fetch data = self.dataset[possibly_batched_index] File "/opt/conda/lib/python3.10/site-packages/gss/utils/data_utils.py", line 49, in getitem self._validate(cuts) File "/opt/conda/lib/python3.10/site-packages/gss/utils/data_utils.py", line 112, in _validate validate(cuts) File "/opt/conda/lib/python3.10/site-packages/lhotse/qa.py", line 39, in validate validator(obj, read_data=read_data) File "/opt/conda/lib/python3.10/site-packages/lhotse/qa.py", line 510, in validate_cut_set validate_cut(c, read_data=read_data) File "/opt/conda/lib/python3.10/site-packages/lhotse/qa.py", line 384, in validate_cut assert is_equal_or_contains(c.recording.channel_ids, c.channel) AssertionError

I found that the audio begin with IS* IN*, and IB* have only 4-channel but not 8-channel in Array2 (ex. https://groups.inf.ed.ac.uk/ami//AMICorpusMirror/amicorpus/IS1000a/audio/). However, egs/ami/ASR/local/prepare_ami_gss.sh stage4 looks to assume that training data has 8-channel audio. I guess the assertion error was caused by this.

I would like to ask you when you implement egs/ami/ASR/pruned_transducer_stateless7, were there 8-channel audio of Array2 in IS* IN*, and IB*?

Thank you.

can-cui commented 1 week ago

Hi, may I ask if you solved this problem, I have the same problem.

hiranoyu0830 commented 1 week ago

Hi, @can-cui I fixed this problem by simply excluding all Array2 in IS* IN*, and IB*. But still waiting for answers from contributors. FYI, the results were almost same as the reported one.

hiranoyu0830 commented 1 week ago

Hello, @desh2608 It seems you are the one who created pruned_transducer_stateless7 on AMI corpus. Do you know anything about the issue above? Thank you.

desh2608 commented 1 week ago

We don't use Array2 at all, so it shouldn't make a difference. The GSS is only applied using Array1.

hiranoyu0830 commented 6 days ago

Thank you very much for your reply!

can-cui commented 6 days ago

Hi, @can-cui I fixed this problem by simply excluding all Array2 in IS* IN*, and IB*. But still waiting for answers from contributors. FYI, the results were almost same as the reported one.

Hi, thank you for the solution! Could you please precise how to choose to only use Array1 in the parameters?

hiranoyu0830 commented 5 days ago

I just deleted Array2.

desh2608 commented 5 days ago

I believe there's also a "--channels" options in the "gss enhance" command. Please refer to the README on the gss repo.

can-cui commented 5 days ago

I believe there's also a "--channels" options in the "gss enhance" command. Please refer to the README on the gss repo.

I just found the reason for my problem: the IS1003b I downloaded didn't have the full Array due to a problem with the data itself. I manually deleted this folder and re-ran the recipe from stage1 of prepare.sh and stage0 of prepare_ami_gss.sh and it ran without error. We don't need to specify Array1.