search

MontrealCorpusTools / Montreal-Forced-Aligner

Command line utility for forced alignment using Kaldi
https://montrealcorpustools.github.io/Montreal-Forced-Aligner/
MIT License
1.29k stars 242 forks source link

[BUG] montreal_forced_aligner.exceptions.DatabaseError: DatabaseError: #637

Closed Moon-sung-woo closed 1 year ago

Moon-sung-woo commented 1 year ago

I don't know why the MFA is over just after the model is trained

Debugging checklist

[o] Have you updated to latest MFA version? (2.2.10) [o] Have you tried rerunning the command with the --clean flag?

General installation (Linux)

conda create -n aligner -c conda-forge montreal-forced-aligner

RUN

mfa train --clean raw_data lexicon.txt ./TextGrid --punctuation "#@$%^" --sample_frequency "22050" --low_frequency "0" high_frequency "8000"

All folders have read and write permissions

For Reproducing your issue Please fill out the following:

  1. Corpus structure
    • What language is the corpus in? Korean
    • How many files/speakers? 35 speakers and 23661 files
    • Are you using lab files or TextGrid files for input? yes i use lib files

Desktop (please complete the following information):

Log file

 INFO     Training model...                                                                                                                                                                                
 INFO     Completed training in 4171.303298711777 seconds!                                                                                                                                                 
 INFO     Saved model to TextGrid                                                                                                                                                                          
 INFO     Done! Everything took 4370.606 seconds                                                                                                                                                           
 INFO     Stopping the global MFA database server...                                                                                                                                                       
 ERROR    pg_ctl stdout: waiting for server to shut down............................................................... failed                                                                             

 ERROR    pg_ctl stderr: pg_ctl: server does not shut down                                                                                                                                                 
          HINT: The "-m fast" option immediately disconnects sessions rather than                                                                                                                          
          waiting for session-initiated disconnection.                                                                                                                                                     

Exception ignored in atexit callback: <function stop_server at 0x7f87e5329e10>
Traceback (most recent call last):
  File "/home/sungwoo/miniconda3/envs/aligner_new/lib/python3.10/site-packages/montreal_forced_aligner/command_line/utils.py", line 470, in stop_server
    raise DatabaseError(
montreal_forced_aligner.exceptions.DatabaseError: DatabaseError:

pg_log_global.txt

2023-05-17 08:11:41.788 KST [1457195] LOG:  starting PostgreSQL 15.3 on x86_64-conda-linux-gnu, compiled by x86_64-conda-linux-gnu-cc (conda-forge gcc 12.2.0-19) 12.2.0, 64-bit
2023-05-17 08:11:41.796 KST [1457195] LOG:  listening on Unix socket "/home/sungwoo/Documents/MFA/pg_mfa_global_socket/.s.PGSQL.5432"
2023-05-17 08:11:41.821 KST [1457227] LOG:  database system was shut down at 2023-05-17 08:11:41 KST
2023-05-17 08:11:41.842 KST [1457195] LOG:  database system is ready to accept connections
2023-05-17 08:11:42.291 KST [1457507] FATAL:  database "DAL-eTTS_v3.0" does not exist
2023-05-17 08:16:41.920 KST [1457225] LOG:  checkpoint starting: time
2023-05-17 08:21:11.516 KST [1457225] LOG:  checkpoint complete: wrote 14257 buffers (43.5%); 0 WAL file(s) added, 0 removed, 18 recycled; write=269.119 s, sync=0.150 s, total=269.597 s; sync files=382, longest=0.010 s, average=0.001 s; distance=298398 kB, estimate=298398 kB
2023-05-17 08:26:36.138 KST [1525322] ERROR:  canceling autovacuum task
2023-05-17 08:26:36.138 KST [1525322] CONTEXT:  automatic vacuum of table "DAL-eTTS_v3.0.public.utterance"
2023-05-17 08:26:41.646 KST [1457225] LOG:  checkpoint starting: time
2023-05-17 08:31:11.176 KST [1457225] LOG:  checkpoint complete: wrote 8174 buffers (24.9%); 0 WAL file(s) added, 0 removed, 8 recycled; write=269.324 s, sync=0.039 s, total=269.530 s; sync files=64, longest=0.011 s, average=0.001 s; distance=130382 kB, estimate=281596 kB
2023-05-17 08:35:24.875 KST [1566684] ERROR:  canceling autovacuum task
2023-05-17 08:35:24.875 KST [1566684] CONTEXT:  automatic vacuum of table "DAL-eTTS_v3.0.public.utterance"
2023-05-17 08:36:41.302 KST [1457225] LOG:  checkpoint starting: time
2023-05-17 08:41:11.128 KST [1457225] LOG:  checkpoint complete: wrote 7556 buffers (23.1%); 0 WAL file(s) added, 0 removed, 5 recycled; write=269.711 s, sync=0.033 s, total=269.826 s; sync files=66, longest=0.008 s, average=0.001 s; distance=79511 kB, estimate=261388 kB
2023-05-17 08:46:41.254 KST [1457225] LOG:  checkpoint starting: time
2023-05-17 08:51:11.134 KST [1457225] LOG:  checkpoint complete: wrote 7750 buffers (23.7%); 0 WAL file(s) added, 0 removed, 4 recycled; write=269.764 s, sync=0.043 s, total=269.880 s; sync files=76, longest=0.010 s, average=0.001 s; distance=73116 kB, estimate=242560 kB
2023-05-17 08:54:31.331 KST [1655140] ERROR:  canceling autovacuum task
2023-05-17 08:54:31.331 KST [1655140] CONTEXT:  automatic vacuum of table "DAL-eTTS_v3.0.public.utterance"
2023-05-17 08:56:41.179 KST [1457225] LOG:  checkpoint starting: time
2023-05-17 09:01:11.180 KST [1457225] LOG:  checkpoint complete: wrote 7828 buffers (23.9%); 0 WAL file(s) added, 0 removed, 8 recycled; write=269.851 s, sync=0.043 s, total=270.002 s; sync files=77, longest=0.010 s, average=0.001 s; distance=122658 kB, estimate=230570 kB
2023-05-17 09:06:08.594 KST [1710459] ERROR:  canceling autovacuum task
2023-05-17 09:06:08.594 KST [1710459] CONTEXT:  automatic vacuum of table "DAL-eTTS_v3.0.public.utterance"
2023-05-17 09:06:41.311 KST [1457225] LOG:  checkpoint starting: time
2023-05-17 09:07:09.538 KST [1715106] ERROR:  canceling autovacuum task
2023-05-17 09:07:09.538 KST [1715106] CONTEXT:  automatic vacuum of table "DAL-eTTS_v3.0.public.utterance"
2023-05-17 09:08:14.965 KST [1719663] ERROR:  canceling autovacuum task
2023-05-17 09:08:14.965 KST [1719663] CONTEXT:  automatic vacuum of table "DAL-eTTS_v3.0.public.utterance"
2023-05-17 09:09:15.094 KST [1724323] ERROR:  canceling autovacuum task
2023-05-17 09:09:15.094 KST [1724323] CONTEXT:  automatic vacuum of table "DAL-eTTS_v3.0.public.utterance"
2023-05-17 09:11:11.119 KST [1457225] LOG:  checkpoint complete: wrote 7712 buffers (23.5%); 0 WAL file(s) added, 0 removed, 5 recycled; write=269.670 s, sync=0.051 s, total=269.809 s; sync files=74, longest=0.011 s, average=0.001 s; distance=88505 kB, estimate=216364 kB
2023-05-17 09:16:41.246 KST [1457225] LOG:  checkpoint starting: time
2023-05-17 09:21:11.225 KST [1457225] LOG:  checkpoint complete: wrote 7269 buffers (22.2%); 0 WAL file(s) added, 0 removed, 10 recycled; write=269.812 s, sync=0.044 s, total=269.979 s; sync files=78, longest=0.011 s, average=0.001 s; distance=156155 kB, estimate=210343 kB
2023-05-17 09:24:28.723 KST [1457225] LOG:  checkpoint starting: wal
2023-05-17 09:24:35.342 KST [1457543] LOG:  duration: 12321.631 ms  statement: COPY "phone_interval" FROM stdin WITH DELIMITER AS ',' NULL AS ''
2023-05-17 09:24:42.953 KST [1457195] LOG:  received smart shutdown request
2023-05-17 09:25:43.295 KST [1457195] LOG:  background worker "logical replication launcher" (PID 1457237) exited with exit code 1
2023-05-17 09:25:43.646 KST [1457225] LOG:  checkpoint complete: wrote 19897 buffers (60.7%); 0 WAL file(s) added, 0 removed, 33 recycled; write=74.579 s, sync=0.045 s, total=74.923 s; sync files=86, longest=0.010 s, average=0.001 s; distance=538842 kB, estimate=538842 kB
2023-05-17 09:25:43.646 KST [1457225] LOG:  shutting down
2023-05-17 09:25:43.654 KST [1457225] LOG:  checkpoint starting: shutdown immediate
2023-05-17 09:25:43.958 KST [1457225] LOG:  checkpoint complete: wrote 8467 buffers (25.8%); 0 WAL file(s) added, 0 removed, 22 recycled; write=0.057 s, sync=0.038 s, total=0.312 s; sync files=33, longest=0.025 s, average=0.002 s; distance=360252 kB, estimate=520983 kB
2023-05-17 09:25:44.013 KST [1457195] LOG:  database system is shut down
2023-05-17 09:38:01.623 KST [1851786] LOG:  starting PostgreSQL 15.3 on x86_64-conda-linux-gnu, compiled by x86_64-conda-linux-gnu-cc (conda-forge gcc 12.2.0-19) 12.2.0, 64-bit
2023-05-17 09:38:01.631 KST [1851786] LOG:  listening on Unix socket "/home/sungwoo/Documents/MFA/pg_mfa_global_socket/.s.PGSQL.5432"
2023-05-17 09:38:01.657 KST [1851796] LOG:  database system was shut down at 2023-05-17 09:25:43 KST
2023-05-17 09:38:01.677 KST [1851786] LOG:  database system is ready to accept connections
2023-05-17 09:38:01.940 KST [1851786] LOG:  received smart shutdown request
2023-05-17 09:38:01.951 KST [1851786] LOG:  background worker "logical replication launcher" (PID 1851799) exited with exit code 1
2023-05-17 09:38:01.951 KST [1851794] LOG:  shutting down
2023-05-17 09:38:01.959 KST [1851794] LOG:  checkpoint starting: shutdown immediate
2023-05-17 09:38:02.016 KST [1851794] LOG:  checkpoint complete: wrote 2 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.018 s, sync=0.006 s, total=0.066 s; sync files=2, longest=0.003 s, average=0.003 s; distance=0 kB, estimate=0 kB
2023-05-17 09:38:02.033 KST [1851786] LOG:  database system is shut down
2023-05-17 09:41:31.088 KST [1866567] LOG:  starting PostgreSQL 15.3 on x86_64-conda-linux-gnu, compiled by x86_64-conda-linux-gnu-cc (conda-forge gcc 12.2.0-19) 12.2.0, 64-bit
2023-05-17 09:41:31.097 KST [1866567] LOG:  listening on Unix socket "/home/sungwoo/Documents/MFA/pg_mfa_global_socket/.s.PGSQL.5432"
2023-05-17 09:41:31.123 KST [1866570] LOG:  database system was shut down at 2023-05-17 09:38:02 KST
2023-05-17 09:41:31.145 KST [1866567] LOG:  database system is ready to accept connections
2023-05-17 09:41:31.398 KST [1866567] LOG:  received smart shutdown request
2023-05-17 09:41:31.416 KST [1866567] LOG:  background worker "logical replication launcher" (PID 1866573) exited with exit code 1
2023-05-17 09:41:31.416 KST [1866568] LOG:  shutting down
2023-05-17 09:41:31.424 KST [1866568] LOG:  checkpoint starting: shutdown immediate
2023-05-17 09:41:31.488 KST [1866568] LOG:  checkpoint complete: wrote 2 buffers (0.0%); 0 WAL file(s) added, 0 removed, 0 recycled; write=0.018 s, sync=0.003 s, total=0.073 s; sync files=2, longest=0.003 s, average=0.002 s; distance=0 kB, estimate=0 kB
2023-05-17 09:41:31.498 KST [1866567] LOG:  database system is shut down
2023-05-17 09:42:31.046 KST [1870784] LOG:  starting PostgreSQL 15.3 on x86_64-conda-linux-gnu, compiled by x86_64-conda-linux-gnu-cc (conda-forge gcc 12.2.0-19) 12.2.0, 64-bit
2023-05-17 09:42:31.054 KST [1870784] LOG:  listening on Unix socket "/home/sungwoo/Documents/MFA/pg_mfa_global_socket/.s.PGSQL.5432"
2023-05-17 09:42:31.079 KST [1870817] LOG:  database system was shut down at 2023-05-17 09:41:31 KST
2023-05-17 09:42:31.097 KST [1870784] LOG:  database system is ready to accept connections
2023-05-17 09:47:31.180 KST [1870815] LOG:  checkpoint starting: time
2023-05-17 09:52:01.325 KST [1870815] LOG:  checkpoint complete: wrote 13481 buffers (41.1%); 0 WAL file(s) added, 17 removed, 1 recycled; write=269.809 s, sync=0.070 s, total=270.145 s; sync files=176, longest=0.009 s, average=0.001 s; distance=294992 kB, estimate=294992 kB
2023-05-17 09:57:19.284 KST [1940274] ERROR:  canceling autovacuum task
2023-05-17 09:57:19.284 KST [1940274] CONTEXT:  automatic vacuum of table "DAL-eTTS_v3.0.public.utterance"
2023-05-17 09:57:31.455 KST [1870815] LOG:  checkpoint starting: time
2023-05-17 10:02:01.139 KST [1870815] LOG:  checkpoint complete: wrote 8361 buffers (25.5%); 0 WAL file(s) added, 2 removed, 6 recycled; write=269.531 s, sync=0.039 s, total=269.685 s; sync files=64, longest=0.011 s, average=0.001 s; distance=135411 kB, estimate=279034 kB
2023-05-17 10:07:31.266 KST [1870815] LOG:  checkpoint starting: time
2023-05-17 10:12:01.140 KST [1870815] LOG:  checkpoint complete: wrote 7853 buffers (24.0%); 0 WAL file(s) added, 2 removed, 3 recycled; write=269.752 s, sync=0.032 s, total=269.874 s; sync files=67, longest=0.008 s, average=0.001 s; distance=81376 kB, estimate=259268 kB
2023-05-17 10:14:29.585 KST [2017735] ERROR:  canceling autovacuum task
2023-05-17 10:14:29.585 KST [2017735] CONTEXT:  automatic vacuum of table "DAL-eTTS_v3.0.public.utterance"
2023-05-17 10:17:31.270 KST [1870815] LOG:  checkpoint starting: time
2023-05-17 10:22:01.156 KST [1870815] LOG:  checkpoint complete: wrote 7538 buffers (23.0%); 0 WAL file(s) added, 3 removed, 2 recycled; write=269.733 s, sync=0.051 s, total=269.886 s; sync files=78, longest=0.010 s, average=0.001 s; distance=78231 kB, estimate=241164 kB
2023-05-17 10:27:31.287 KST [1870815] LOG:  checkpoint starting: time
2023-05-17 10:32:01.191 KST [1870815] LOG:  checkpoint complete: wrote 7829 buffers (23.9%); 0 WAL file(s) added, 0 removed, 7 recycled; write=269.748 s, sync=0.043 s, total=269.905 s; sync files=76, longest=0.011 s, average=0.001 s; distance=126435 kB, estimate=229691 kB
2023-05-17 10:36:37.771 KST [2118124] ERROR:  canceling autovacuum task
2023-05-17 10:36:37.771 KST [2118124] CONTEXT:  automatic vacuum of table "DAL-eTTS_v3.0.public.utterance"
2023-05-17 10:37:31.318 KST [1870815] LOG:  checkpoint starting: time
2023-05-17 10:37:38.493 KST [2122755] ERROR:  canceling autovacuum task
2023-05-17 10:37:38.493 KST [2122755] CONTEXT:  automatic vacuum of table "DAL-eTTS_v3.0.public.utterance"
2023-05-17 10:38:44.331 KST [2127285] ERROR:  canceling autovacuum task
2023-05-17 10:38:44.331 KST [2127285] CONTEXT:  automatic vacuum of table "DAL-eTTS_v3.0.public.utterance"
2023-05-17 10:39:44.117 KST [2131916] ERROR:  canceling autovacuum task
2023-05-17 10:39:44.117 KST [2131916] CONTEXT:  automatic vacuum of table "DAL-eTTS_v3.0.public.utterance"
2023-05-17 10:42:01.161 KST [1870815] LOG:  checkpoint complete: wrote 7684 buffers (23.4%); 0 WAL file(s) added, 3 removed, 3 recycled; write=269.712 s, sync=0.033 s, total=269.843 s; sync files=68, longest=0.008 s, average=0.001 s; distance=89703 kB, estimate=215692 kB
2023-05-17 10:42:31.190 KST [1870815] LOG:  checkpoint starting: time
2023-05-17 10:47:01.175 KST [1870815] LOG:  checkpoint complete: wrote 6227 buffers (19.0%); 0 WAL file(s) added, 0 removed, 9 recycled; write=269.824 s, sync=0.044 s, total=269.985 s; sync files=79, longest=0.010 s, average=0.001 s; distance=158813 kB, estimate=210004 kB
2023-05-17 10:52:31.259 KST [1870815] LOG:  checkpoint starting: time
2023-05-17 10:52:38.684 KST [1870815] LOG:  checkpoint complete: wrote 75 buffers (0.2%); 0 WAL file(s) added, 0 removed, 0 recycled; write=7.349 s, sync=0.035 s, total=7.425 s; sync files=45, longest=0.010 s, average=0.001 s; distance=315 kB, estimate=189035 kB
2023-05-17 10:53:15.491 KST [2195212] ERROR:  canceling autovacuum task
2023-05-17 10:53:15.491 KST [2195212] CONTEXT:  automatic vacuum of table "DAL-eTTS_v3.0.public.utterance"
2023-05-17 10:54:23.978 KST [2199864] ERROR:  canceling autovacuum task
2023-05-17 10:54:23.978 KST [2199864] CONTEXT:  automatic vacuum of table "DAL-eTTS_v3.0.public.utterance"
2023-05-17 10:54:56.913 KST [1870815] LOG:  checkpoint starting: wal
2023-05-17 10:55:03.795 KST [1871263] LOG:  duration: 12131.705 ms  statement: COPY "phone_interval" FROM stdin WITH DELIMITER AS ',' NULL AS ''
2023-05-17 10:55:11.550 KST [1870784] LOG:  received smart shutdown request
2023-05-17 10:56:11.864 KST [1870784] LOG:  background worker "logical replication launcher" (PID 1870825) exited with exit code 1
2023-05-17 10:56:12.242 KST [1870815] LOG:  checkpoint complete: wrote 15237 buffers (46.5%); 0 WAL file(s) added, 0 removed, 33 recycled; write=74.931 s, sync=0.086 s, total=75.372 s; sync files=77, longest=0.017 s, average=0.002 s; distance=532725 kB, estimate=532725 kB
2023-05-17 10:56:12.243 KST [1870815] LOG:  shutting down
2023-05-17 10:56:12.251 KST [1870815] LOG:  checkpoint starting: shutdown immediate
2023-05-17 10:56:12.605 KST [1870815] LOG:  checkpoint complete: wrote 12215 buffers (37.3%); 0 WAL file(s) added, 0 removed, 23 recycled; write=0.089 s, sync=0.036 s, total=0.363 s; sync files=46, longest=0.018 s, average=0.001 s; distance=376830 kB, estimate=517135 kB
2023-05-17 10:56:12.659 KST [1870784] LOG:  database system is shut down
Moon-sung-woo commented 1 year ago

@mmcauliffe Hi. I updated MFA version. But i still have this error.

yuxi-chen19 commented 1 year ago

+1