[x] updated to latest MFA version
[x] running the command with the --clean flag
Describe the issue
I installed the newest version of the MFA with conda and also the newest version of SQlite (3.42.0). I have downloaded the german_mfa dictionary and acoustic model in the corresponding folders under MFA. I activated the aligner environment, set up the paths to my input and output and used the correct paths to all relevant parameters. In the beginning it seems everything is working fine but the process always stopps with this error message:
subprocess.CalledProcessError: Command '['sqlite3', 'C:/Users/Andrea Hofmann/Documents/MFA/input/input.db', '--cmd', '.mode csv', '.import C:/Users/Andrea Hofmann/Documents/MFA/input/alignment/word_intervals.csv word_interval_temp']' returned non-zero exit status 1.
Somehow subprocess call to SQLite is not completing successfully, and it seems to be due to an extra argument "word_interval_temp" which is not recognized by SQLite.
I have no idea what to do.
For Reproducing your issue
Please fill out the following:
Corpus structure
What language is the corpus in?
German
How many files/speakers?
20 files, 1 speaker
Are you using lab files or TextGrid files for input?
each of the 20 wav files has a TextGrid with the same name
Dictionary
Are you using a dictionary from MFA? If so, which one?
I downloaded and use: german_mfa
If it's a custom dictionary, what is the phoneset?
Acoustic model
If you're using an acoustic model, is it one download through MFA? If so, which one?
german_mfa as well
Log file
Please attach the log file for the run that encountered an error (by default these will be stored in ~/Documents/MFA).
Desktop (please complete the following information):
OS: Windows
Version: Windows 10
Any other details about the setup: the env is called aligner, there is a space in my paths that I can't get rid off, but I use quotation marks, I had to install sqlite and used the newest version 3.42.0
Additional context
I can see that most of the data is there in the input folder but I can't get the process to finish.
This is the miniconda console output, if it helps:
compile_train_graphs.1.logget_phone_ctm.1.logalign.1.fmllr.logalign.1.logcalc_fmllr.1.log
This is the miniconda console output, if it helps:
(aligner) C:\Users\Andrea Hofmann>mfa align --clean "C:\Users\Andrea Hofmann\OneDrive\PhD\exp_dualtask\data_pre-processing\MFA\input" "C:\Users\Andrea Hofmann\Documents\MFA\pretrained_models\dictionary\german_mfa.dict" "C:\Users\Andrea Hofmann\Documents\MFA\pretrained_models\acoustic\german_mfa.zip" "C:\Users\Andrea Hofmann\OneDrive\PhD\exp_dualtask\data_pre-processing\MFA\ouput"
DEBUG Beginning run for input
DEBUG Using "global" profile
DEBUG Using multiprocessing with 3
DEBUG Set up logger for MFA version: 2.2.15
DEBUG Cleaned previous run
DEBUG There were some differences in the current run compared to the last one. This may cause issues, run with
--clean, if you hit an error.
DEBUG Using IPA
DEBUG Loaded dictionary in 44.509 seconds
INFO Setting up corpus information...
DEBUG Could not load from temp
INFO Loading corpus from source files...
1% ╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/100 [ 0:00:05 < -:--:-- , ? it/s ] DEBUG Processing queue: 0.140625
1% ╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/100 [ 0:00:05 < -:--:-- , ? it/s ]
DEBUG Parsed corpus directory with 3 jobs in 0.15625 seconds
INFO Found 1 speaker across 20 files, average number of utterances per speaker: 20.0
DEBUG Loaded corpus in 5.839 seconds
INFO Initializing multiprocessing jobs...
WARNING Number of jobs was specified as 3, but due to only having 1 speakers, MFA will only use 1 jobs. Use the
--single_speaker flag if you would like to split utterances across jobs regardless of their speaker.
DEBUG Initialized jobs in 0.029 seconds
INFO Normalizing text...
5% ━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/20 [ 0:00:11 < -:--:-- , ? it/s ]
DEBUG Wrote lexicon information in 0.376 seconds
INFO Creating corpus split for feature generation...
2% ━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/40 [ 0:00:04 < -:--:-- , ? it/s ]
DEBUG Created corpus split directory in 4.566 seconds
INFO Generating MFCCs...
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20/20 [ 0:00:05 < 0:00:00 , 61 it/s ]
DEBUG Generating MFCCs took 6.781 seconds
INFO Calculating CMVN...
INFO Generating final features...
5% ━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/20 [ 0:00:05 < -:--:-- , ? it/s ]
DEBUG Generating final features took 5.755 seconds
INFO Creating corpus split with features...
5% ━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/20 [ 0:00:04 < -:--:-- , ? it/s ]
DEBUG Generated features in 17.039 seconds
DEBUG Setting up corpus took 84.110 seconds
DEBUG
DEBUG ====ACOUSTIC MODEL INFO====
DEBUG Acoustic model root directory: C:\Users\Andrea Hofmann\Documents\MFA\extracted_models\acoustic
DEBUG Acoustic model dirname: C:\Users\Andrea Hofmann\Documents\MFA\extracted_models\acoustic\german_mfa_acoustic
DEBUG Acoustic model meta path: C:\Users\Andrea
Hofmann\Documents\MFA\extracted_models\acoustic\german_mfa_acoustic\meta.json
DEBUG Acoustic model meta information:
DEBUG architecture: gmm-hmm
dictionaries:
bracketed_word: '[bracketed]'
clitic_marker: ''''
default: german_mfa
laughter_word: '[laughter]'
names:
DEBUG
DEBUG Setup for alignment in 84.239 seconds
INFO Compiling training graphs...
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20/20 [ 0:00:05 < 0:00:00 , ? it/s ]
DEBUG Compiling training graphs took 6.601 seconds
INFO Performing first-pass alignment...
INFO Generating alignments...
85% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━ 17/20 [ 0:00:07 < 0:00:01 , 79 it/s ]
DEBUG Alignment round took 7.238 seconds
INFO Calculating fMLLR for speaker adaptation...
100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/1 [ 0:00:04 < 0:00:00 , ? it/s ]
DEBUG Fmllr calculation took 5.873 seconds
INFO Performing second-pass alignment...
INFO Generating alignments...
75% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━ 15/20 [ 0:00:07 < 0:00:01 , 128 it/s ]
DEBUG Alignment round took 7.317 seconds
INFO Collecting phone and word alignments from alignment lattices...
5% ━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/20 [ 0:00:11 < -:--:-- , ? it/s ]ERROR: extra argument: "word_interval_temp". Usage:
.import FILE TABLE Import data from FILE into TABLE
Options:
--ascii Use \037 and \036 as column and row separators
--csv Use , and \n as column and row separators
--skip N Skip the first N rows of input
--schema S Target table to be S.TABLE
-v "Verbose" - increase auxiliary output
Notes:
If TABLE does not exist, it is created. The first row of input
determines the column names.
If neither --csv or --ascii are used, the input mode is derived
from the ".mode" output mode
If FILE begins with "|" then it is a command that generates the
input text.
5% ━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/20 [ 0:00:11 < -:--:-- , ? it/s ]
ERROR There was an error in the run, please see the log.
Exception ignored in atexit callback: <bound method ExitHooks.history_save_handler of <montreal_forced_aligner.command_line.mfa.ExitHooks object at 0x00000232A1D433A0>>
Traceback (most recent call last):
File "C:\Apps\miniconda\envs\aligner\lib\site-packages\montreal_forced_aligner\command_line\mfa.py", line 98, in history_save_handler
raise self.exception
File "C:\Apps\miniconda\envs\aligner\Scripts\mfa-script.py", line 9, in
sys.exit(mfa_cli())
File "C:\Apps\miniconda\envs\aligner\lib\site-packages\click\core.py", line 1157, in call
return self.main(*args, kwargs)
File "C:\Apps\miniconda\envs\aligner\lib\site-packages\rich_click\rich_group.py", line 21, in main
rv = super().main(args, standalone_mode=False, kwargs)
File "C:\Apps\miniconda\envs\aligner\lib\site-packages\click\core.py", line 1078, in main
rv = self.invoke(ctx)
File "C:\Apps\miniconda\envs\aligner\lib\site-packages\click\core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Apps\miniconda\envs\aligner\lib\site-packages\click\core.py", line 1434, in invoke
return ctx.invoke(self.callback, ctx.params)
File "C:\Apps\miniconda\envs\aligner\lib\site-packages\click\core.py", line 783, in invoke
return __callback(args, kwargs)
File "C:\Apps\miniconda\envs\aligner\lib\site-packages\click\decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "C:\Apps\miniconda\envs\aligner\lib\site-packages\montreal_forced_aligner\command_line\align.py", line 113, in align_corpus_cli
aligner.align()
File "C:\Apps\miniconda\envs\aligner\lib\site-packages\montreal_forced_aligner\alignment\pretrained.py", line 413, in align
super().align()
File "C:\Apps\miniconda\envs\aligner\lib\site-packages\montreal_forced_aligner\alignment\base.py", line 365, in align
self.collect_alignments()
File "C:\Apps\miniconda\envs\aligner\lib\site-packages\montreal_forced_aligner\alignment\base.py", line 923, in collect_alignments
subprocess.check_call(
File "C:\Apps\miniconda\envs\aligner\lib\subprocess.py", line 369, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['sqlite3', 'C:/Users/Andrea Hofmann/Documents/MFA/input/input.db', '--cmd', '.mode csv', '.import C:/Users/Andrea Hofmann/Documents/MFA/input/alignment/word_intervals.csv word_interval_temp']' returned non-zero exit status 1.
Okay, I uninstalled everything again and this time I set up 'MFA_ROOT_DIR' with a path that did not include spaces and ran the aligner again and it worked. Uff.
Debugging checklist
[x] updated to latest MFA version [x] running the command with the
--clean
flagDescribe the issue I installed the newest version of the MFA with conda and also the newest version of SQlite (3.42.0). I have downloaded the german_mfa dictionary and acoustic model in the corresponding folders under MFA. I activated the aligner environment, set up the paths to my input and output and used the correct paths to all relevant parameters. In the beginning it seems everything is working fine but the process always stopps with this error message: subprocess.CalledProcessError: Command '['sqlite3', 'C:/Users/Andrea Hofmann/Documents/MFA/input/input.db', '--cmd', '.mode csv', '.import C:/Users/Andrea Hofmann/Documents/MFA/input/alignment/word_intervals.csv word_interval_temp']' returned non-zero exit status 1. Somehow subprocess call to SQLite is not completing successfully, and it seems to be due to an extra argument "word_interval_temp" which is not recognized by SQLite. I have no idea what to do.
For Reproducing your issue Please fill out the following:
Log file Please attach the log file for the run that encountered an error (by default these will be stored in
~/Documents/MFA
).Desktop (please complete the following information):
Additional context I can see that most of the data is there in the input folder but I can't get the process to finish. This is the miniconda console output, if it helps: compile_train_graphs.1.log get_phone_ctm.1.log align.1.fmllr.log align.1.log calc_fmllr.1.log This is the miniconda console output, if it helps: (aligner) C:\Users\Andrea Hofmann>mfa align --clean "C:\Users\Andrea Hofmann\OneDrive\PhD\exp_dualtask\data_pre-processing\MFA\input" "C:\Users\Andrea Hofmann\Documents\MFA\pretrained_models\dictionary\german_mfa.dict" "C:\Users\Andrea Hofmann\Documents\MFA\pretrained_models\acoustic\german_mfa.zip" "C:\Users\Andrea Hofmann\OneDrive\PhD\exp_dualtask\data_pre-processing\MFA\ouput" DEBUG Beginning run for input DEBUG Using "global" profile DEBUG Using multiprocessing with 3 DEBUG Set up logger for MFA version: 2.2.15 DEBUG Cleaned previous run DEBUG There were some differences in the current run compared to the last one. This may cause issues, run with --clean, if you hit an error. DEBUG Using IPA DEBUG Loaded dictionary in 44.509 seconds INFO Setting up corpus information... DEBUG Could not load from temp INFO Loading corpus from source files... 1% ╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/100 [ 0:00:05 < -:--:-- , ? it/s ] DEBUG Processing queue: 0.140625 1% ╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/100 [ 0:00:05 < -:--:-- , ? it/s ] DEBUG Parsed corpus directory with 3 jobs in 0.15625 seconds INFO Found 1 speaker across 20 files, average number of utterances per speaker: 20.0 DEBUG Loaded corpus in 5.839 seconds INFO Initializing multiprocessing jobs... WARNING Number of jobs was specified as 3, but due to only having 1 speakers, MFA will only use 1 jobs. Use the --single_speaker flag if you would like to split utterances across jobs regardless of their speaker. DEBUG Initialized jobs in 0.029 seconds INFO Normalizing text... 5% ━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/20 [ 0:00:11 < -:--:-- , ? it/s ] DEBUG Wrote lexicon information in 0.376 seconds INFO Creating corpus split for feature generation... 2% ━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/40 [ 0:00:04 < -:--:-- , ? it/s ] DEBUG Created corpus split directory in 4.566 seconds INFO Generating MFCCs... 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20/20 [ 0:00:05 < 0:00:00 , 61 it/s ] DEBUG Generating MFCCs took 6.781 seconds INFO Calculating CMVN... INFO Generating final features... 5% ━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/20 [ 0:00:05 < -:--:-- , ? it/s ] DEBUG Generating final features took 5.755 seconds INFO Creating corpus split with features... 5% ━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/20 [ 0:00:04 < -:--:-- , ? it/s ] DEBUG Generated features in 17.039 seconds DEBUG Setting up corpus took 84.110 seconds DEBUG DEBUG ====ACOUSTIC MODEL INFO==== DEBUG Acoustic model root directory: C:\Users\Andrea Hofmann\Documents\MFA\extracted_models\acoustic DEBUG Acoustic model dirname: C:\Users\Andrea Hofmann\Documents\MFA\extracted_models\acoustic\german_mfa_acoustic DEBUG Acoustic model meta path: C:\Users\Andrea Hofmann\Documents\MFA\extracted_models\acoustic\german_mfa_acoustic\meta.json DEBUG Acoustic model meta information: DEBUG architecture: gmm-hmm dictionaries: bracketed_word: '[bracketed]' clitic_marker: '''' default: german_mfa laughter_word: '[laughter]' names:
german_mfa oov_word:
position_dependent_phones: true
silence_word:
use_g2p: false
features:
allow_downsample: true
allow_upsample: true
delta_pitch: 0.005
feature_type: mfcc
frame_length: 25
frame_shift: 10
high_frequency: 7800
low_frequency: 20
max_f0: 500
min_f0: 50
penalty_factor: 0.1
sample_frequency: 16000
snip_edges: true
use_delta_pitch: true
use_energy: false
use_pitch: true
use_voicing: true
uses_cmvn: true
uses_deltas: false
uses_speaker_adaptation: true
uses_splices: true
uses_voiced: false
final_non_silence_correction: 0.04
final_silence_correction: 2.32
initial_silence_probability: 0.2
oov_phone: spn
optional_silence_phone: sil
phone_set_type: IPA
phone_type: triphone
phones: !!set
a: null
aj: null
aw: null
"a\u02D0": null
b: null
c: null
"c\u02B0": null
d: null
e: null
"e\u02D0": null
f: null
h: null
i: null
"i\u02D0": null
j: null
k: null
"k\u02B0": null
l: null
"l\u0329": null
m: null
"m\u0329": null
n: null
"n\u0329": null
o: null
"o\u02D0": null
p: null
pf: null
"p\u02B0": null
s: null
t: null
ts: null
"t\u0283": null
"t\u02B0": null
u: null
"u\u02D0": null
v: null
x: null
y: null
"y\u02D0": null
z: null
"\xE7": null
"\xF8": null
"\xF8\u02D0": null
"\u014B": null
"\u0153": null
"\u0250": null
"\u0254": null
"\u0254\u028F": null
"\u0259": null
"\u025B": null
"\u025F": null
"\u0261": null
"\u026A": null
"\u0272": null
"\u0281": null
"\u0283": null
"\u028A": null
"\u028F": null
silence_probability: 0.19707377600811818
train_date: '2022-05-20 18:27:23.975385'
training:
audio_duration: 11030325.592381079
average_log_likelihood: -0.007585279091018096
num_oovs: 0
num_speakers: 16686
num_utterances: 1231261
version: 2.0.0rc4.dev19+ged818cb.d20220404
DEBUG DEBUG Setup for alignment in 84.239 seconds INFO Compiling training graphs... 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 20/20 [ 0:00:05 < 0:00:00 , ? it/s ] DEBUG Compiling training graphs took 6.601 seconds INFO Performing first-pass alignment... INFO Generating alignments... 85% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━ 17/20 [ 0:00:07 < 0:00:01 , 79 it/s ] DEBUG Alignment round took 7.238 seconds INFO Calculating fMLLR for speaker adaptation... 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/1 [ 0:00:04 < 0:00:00 , ? it/s ] DEBUG Fmllr calculation took 5.873 seconds INFO Performing second-pass alignment... INFO Generating alignments... 75% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╸━━━━━━━━━━━━━━━━━━ 15/20 [ 0:00:07 < 0:00:01 , 128 it/s ] DEBUG Alignment round took 7.317 seconds INFO Collecting phone and word alignments from alignment lattices... 5% ━━━╸━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/20 [ 0:00:11 < -:--:-- , ? it/s ]ERROR: extra argument: "word_interval_temp". Usage: .import FILE TABLE Import data from FILE into TABLE Options: --ascii Use \037 and \036 as column and row separators --csv Use , and \n as column and row separators --skip N Skip the first N rows of input --schema S Target table to be S.TABLE -v "Verbose" - increase auxiliary output Notes: