[BUG]

[diegotg2000]

Debugging checklist

[x] Have you updated to latest MFA version? [x] Have you tried rerunning the command with the --clean flag?

Describe the issue When validating the data I get a syntax error of a SQL query sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) near "~": syntax error [SQL: UPDATE utterance SET in_subset=? WHERE utterance.id IN (SELECT anon_1.id FROM (SELECT utterance.id AS id FROM utterance WHERE (utterance.text ~ ?) AND utterance.ignored = 0 ORDER BY utterance.duration LIMIT ? OFFSET ?) AS anon_1 ORDER BY random() LIMIT ? OFFSET ?) RETURNING id] [parameters: (1, '\s\S+\s', 20000, 0, 2000, 0)]

For Reproducing your issue Please fill out the following:

1. Corpus structure
   • What language is the corpus in? Nigerian pidgin
   • How many files/speakers? 75 speakers, 99 utterances per speaker
   • Are you using lab files or TextGrid files for input? lab files
2. Dictionary
   • Are you using a dictionary from MFA? If so, which one? No
   • If it's a custom dictionary, what is the phoneset? I use a dummy dictionary in which I map the filename of the utterance with the phonetic transcription. The phonemes are encoded using the SAMPA alphabet.
3. Acoustic model
   • If you're using an acoustic model, is it one download through MFA? If so, which one?
   • If it's a model you've trained, what data was it trained on?

Log file pg_log_global.txt

Desktop (please complete the following information):

• OS: Linux
• Version Ubuntu 22.04
• Any other details about the setup (Cloud, Docker, etc)

Additional context My goal is to train and then align the corpus. The error comes from a query executed at line 1165 of the montreal_forced_aligner/corpus/base.py script.

[mfaytak]

I am getting the same error message off a fresh conda install on Mac OS (Ventura 13.4).

Debugging checklist

[x] Have you updated to latest MFA version? [x] Have you tried rerunning the command with the --clean flag?

Issue running mfa validate [my corpus] [my dictionary] fails with an SQL Alchemy (?) query as follows:

sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) near "~": syntax error
[SQL: UPDATE utterance SET in_subset=? WHERE utterance.id IN (SELECT anon_1.id 
FROM (SELECT utterance.id AS id 
FROM utterance 
WHERE (utterance.text ~ ?) AND utterance.ignored = 0 ORDER BY utterance.duration
 LIMIT ? OFFSET ?) AS anon_1 ORDER BY random()
 LIMIT ? OFFSET ?) RETURNING id]
[parameters: (1, '\\s\\S+\\s', 20000, 0, 2000, 0)]
(Background on this error at: https://sqlalche.me/e/20/e3q8)

Full command line output and traceback leading to this: full-traceback.txt

The failure seems to come when the mfa validate routine is meant to create the subset directory of 2000 utterances.

For Reproducing your issue Please fill out the following:

Corpus structure

• What language is the corpus in? Kom
• How many files/speakers? 17 speakers, ~350 utterances per speaker, 27 files (mix of mono and stereo)
• Are you using lab files or TextGrid files for input? TextGrid files

Dictionary

• Are you using a dictionary from MFA? If so, which one? No; self-generated
• If it's a custom dictionary, what is the phoneset? i i0 e eh a a0 o u ɨ ɨ0 ae oe ue ay ey oy uy ɨy zɨ vɨ b nb m nm f nf w nw t nt d nd s ns n nn l nl ch nch j nj y nny k nk g ng ŋ gh '

Acoustic model

• If you're using an acoustic model, is it one download through MFA? If so, which one? No pretrained model exists; the goal is to train an acoustic model and then align with it.
• If it's a model you've trained, what data was it trained on? n/a

Log file

pg_log_global.txt (contains a few runs including the --clean run)

Desktop (please complete the following information):

• OS: Mac OS
• Version: Ventura, 13.4
• Any other details about the setup (Cloud, Docker, etc) n/a

[mfaytak]

Thanks! I am a bit clueless on how this bug fix will be distributed, and need to use the aligner right away - will a conda update get this onto my device?

[mmcauliffe]

It should trickle through the conda pipeline this evening, probably like 1-2 hours, and then you can run conda update montreal-forced-aligner to pick it up.

[mmcauliffe]

You can also rerun with the mfa validate ... --use_postgres --auto_server and that will run the aligner using the PostgreSQL backend instead of sqlite that won't be affected by this bug.

[hanspaa2017108]

You can also rerun with the mfa validate ... --use_postgres --auto_server and that will run the aligner using the PostgreSQL backend instead of sqlite that won't be affected by this bug.

Can this be used with Mfa align command as well ?? @mmcauliffe