search

MontrealCorpusTools / Montreal-Forced-Aligner

Command line utility for forced alignment using Kaldi
https://montrealcorpustools.github.io/Montreal-Forced-Aligner/
MIT License
1.26k stars 242 forks source link

[BUG] RuntimeError: kaldi::KaldiFatalError #689

Open ninackjeong opened 10 months ago

ninackjeong commented 10 months ago

Debugging checklist

[ O ] Have you updated to latest MFA version? [ O ] Have you tried rerunning the command with the --clean flag?

Describe the issue A clear and concise description of what the bug is.

For Reproducing your issue Please fill out the following:

  1. Corpus structure
    • What language is the corpus in? Korean
    • How many files/speakers? 2 people (but have a lot of utterances and have a lot of speakers' files - this is just for testing)
    • Are you using lab files or TextGrid files for input? lab files
  2. Dictionary
    • Are you using a dictionary from MFA? If so, which one? No. I just created it with the sample file
    • If it's a custom dictionary, what is the phoneset? Well, it just consists of jamonized Korean
  3. Acoustic model
    • If you're using an acoustic model, is it one download through MFA? If so, which one? g2p
    • If it's a model you've trained, what data was it trained on? A Korean dialogue between two Korean people

Log file Please attach the log file for the run that encountered an error (by default these will be stored in ~/Documents/MFA).

(aligner) Cheonkams-MacBook-Pro:output ninackjeong$ mfa train --clean /Volumes/ssd/dissertation/scripts/tono-init-spon/sample-test/data/sound/SDRW2100000003_wav/ /Volumes/ssd/dissertation/scripts/tono-init-spon/sample-test/data/output/korean.txt /Volumes/ssd/dissertation/scripts/tono-init-spon/sample-test/data/output/textgrids/
 INFO     Setting up corpus information...                                                  
 INFO     Loading corpus from source files...                                               
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/100  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Found 1 speaker across 219 files, average number of utterances per speaker: 219.0 
 INFO     Initializing multiprocessing jobs...                                              
 WARNING  Number of jobs was specified as 3, but due to only having 1 speakers, MFA will    
          only use 1 jobs. Use the --single_speaker flag if you would like to split         
          utterances across jobs regardless of their speaker.                               
 INFO     Normalizing text...                                                               
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Generating MFCCs...                                                               
  84% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╺━━━━━━ 185/219  [ 0:00:02 < 0:00:01 , 455 it/s ]
 INFO     Calculating CMVN...                                                               
 INFO     Generating final features...                                                      
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 WARNING  There were 1 utterances ignored due to an issue in feature generation, see the log
          file for full details or run `mfa validate` on the corpus.                        
 INFO     Creating corpus split...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Filtering utterances for training...                                              
 INFO     Initializing training for monophone...                                            
 INFO     Compiling training graphs...                                                      
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Generating initial alignments...                                                  
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Initialization complete!                                                          
 INFO     monophone - Iteration 1 of 40                                                     
 INFO     Generating alignments...                                                          
   6% ━━━╺━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 2 of 40                                                     
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 3 of 40                                                     
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 4 of 40                                                     
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 5 of 40                                                     
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 6 of 40                                                     
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 7 of 40                                                     
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 8 of 40                                                     
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 9 of 40                                                     
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 10 of 40                                                    
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 11 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 12 of 40                                                    
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 13 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 14 of 40                                                    
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 15 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 16 of 40                                                    
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 17 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 18 of 40                                                    
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 19 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 20 of 40                                                    
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 21 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 22 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 23 of 40                                                    
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 24 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 25 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 26 of 40                                                    
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 27 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 28 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 29 of 40                                                    
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 30 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 31 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 32 of 40                                                    
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 33 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 34 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 35 of 40                                                    
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 36 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 37 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 38 of 40                                                    
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 39 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     monophone - Iteration 40 of 40                                                    
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Training complete!                                                                
 INFO     Compiling training graphs...                                                      
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Initializing training for triphone...                                             
 INFO     Accumulating tree stats...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Compiling training graphs...                                                      
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Converting alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Initialization complete!                                                          
 INFO     triphone - Iteration 1 of 35                                                      
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 2 of 35                                                      
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 3 of 35                                                      
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 4 of 35                                                      
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 5 of 35                                                      
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 6 of 35                                                      
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 7 of 35                                                      
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 8 of 35                                                      
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 9 of 35                                                      
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 10 of 35                                                     
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 11 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 12 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 13 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 14 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 15 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 16 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 17 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 18 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 19 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 20 of 35                                                     
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 21 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 22 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 23 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 24 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 25 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 26 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 27 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 28 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 29 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 30 of 35                                                     
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 31 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 32 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 33 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 34 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     triphone - Iteration 35 of 35                                                     
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Training complete!                                                                
 INFO     Compiling training graphs...                                                      
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Generating alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Initializing training for lda...                                                  
 INFO     Calculating initial LDA stats...                                                  
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Accumulating tree stats...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Compiling training graphs...                                                      
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Converting alignments...                                                          
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     Initialization complete!                                                          
 INFO     lda - Iteration 1 of 35                                                           
 INFO     Accumulating statistics...                                                        
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 INFO     lda - Iteration 2 of 35                                                           
 INFO     Re-calculating LDA...                                                             
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 0/219  [ 0:00:01 < -:--:-- , ? it/s ]
 ERROR    There was an error in the run, please see the log.                                
Exception ignored in atexit callback: <bound method ExitHooks.history_save_handler of <montreal_forced_aligner.command_line.mfa.ExitHooks object at 0x1add59c90>>
Traceback (most recent call last):
  File "/Users/ninackjeong/miniconda3/envs/aligner/lib/python3.11/site-packages/montreal_forced_aligner/command_line/mfa.py", line 99, in history_save_handler
    raise self.exception
  File "/Users/ninackjeong/miniconda3/envs/aligner/bin/mfa", line 10, in <module>
    sys.exit(mfa_cli())
             ^^^^^^^^^
  File "/Users/ninackjeong/miniconda3/envs/aligner/lib/python3.11/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ninackjeong/miniconda3/envs/aligner/lib/python3.11/site-packages/rich_click/rich_group.py", line 21, in main
    rv = super().main(*args, standalone_mode=False, **kwargs)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ninackjeong/miniconda3/envs/aligner/lib/python3.11/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
         ^^^^^^^^^^^^^^^^
  File "/Users/ninackjeong/miniconda3/envs/aligner/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ninackjeong/miniconda3/envs/aligner/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ninackjeong/miniconda3/envs/aligner/lib/python3.11/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ninackjeong/miniconda3/envs/aligner/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
    return f(get_current_context(), *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/ninackjeong/miniconda3/envs/aligner/lib/python3.11/site-packages/montreal_forced_aligner/command_line/train_acoustic_model.py", line 117, in train_acoustic_model_cli
    trainer.train()
  File "/Users/ninackjeong/miniconda3/envs/aligner/lib/python3.11/site-packages/montreal_forced_aligner/acoustic_modeling/trainer.py", line 523, in train
    trainer.train()
  File "/Users/ninackjeong/miniconda3/envs/aligner/lib/python3.11/site-packages/montreal_forced_aligner/acoustic_modeling/base.py", line 390, in train
    self.train_iteration()
  File "/Users/ninackjeong/miniconda3/envs/aligner/lib/python3.11/site-packages/montreal_forced_aligner/acoustic_modeling/lda.py", line 432, in train_iteration
    self.calc_lda_mllt()
  File "/Users/ninackjeong/miniconda3/envs/aligner/lib/python3.11/site-packages/montreal_forced_aligner/acoustic_modeling/lda.py", line 412, in calc_lda_mllt
    acoustic_model.transform_means(mat)
RuntimeError: kaldi::KaldiFatalError

Desktop (please complete the following information):

Additional context Add any other context about the problem here.

mmcauliffe commented 10 months ago

At first glance I can't say for sure what's going wrong. Can you try installing 2.2.17 via conda install montreal-forced-aligner=2.2.17 and see if the same mfa train command completes?

ninackjeong commented 10 months ago

@mmcauliffe I installed the aligner a few days ago. Isn't it 2.2.17? Of course I am going to test it though.

mmcauliffe commented 10 months ago

No, there's an alpha version of 3.0 out which is what this looks like. Unfortunately, conda only allows for distribution of one version, so I can't do a stable and latest split, so you have to specify older versions. I'm not sure if the errors you're seeing are caused by 3.0 or if they're an issue with the corpus that would pop up on any version.

ninackjeong commented 10 months ago
Screen Shot 2023-09-09 at 9 22 51 PM

I got this error...

mmcauliffe commented 10 months ago

Ah sorry, conda install -c conda-forge montreal-forced-aligner=2.2.17 should work

ninackjeong commented 10 months ago
Screen Shot 2023-09-09 at 11 41 32 PM

That version is way better than the previous one that I used, thanks! I got another error... What is "Ida"? Is it a folder that comes to a being while training?

I am so sorry to bother you, but I should do this...

mmcauliffe commented 10 months ago

Hmm, can you try rerunning the mfa train command with --clean and see if that works? If that still doesn't work, can you make a google drive folder of the files if you can share them with me and send a link to michael.e.mcauliffe@gmail.com and I can try reproducing and see what's happening.

Jueun0505 commented 7 months ago

Ah sorry, conda install -c conda-forge montreal-forced-aligner=2.2.17 should work

Regarding this issue, I think the current documentation needs to be updated for 'Intalling older versions of MFA' (https://montreal-forced-aligner.readthedocs.io/en/latest/installation.html) with the code you suggested above (adding '-c') because I also had the installing issue. Many thanks