search

MontrealCorpusTools / Montreal-Forced-Aligner

Command line utility for forced alignment using Kaldi
https://montrealcorpustools.github.io/Montreal-Forced-Aligner/
MIT License
1.31k stars 244 forks source link

[BUG] FATAL: SetFlags: Bad option #753

Closed stefanocoretta closed 7 months ago

stefanocoretta commented 7 months ago

Debugging checklist

[ ] Have you updated to latest MFA version? Tried 3.0.0a8, but error, so tried latest stable 2.2.17 [x] Have you tried rerunning the command with the --clean flag?

Describe the issue

I am running mfa validate on a corpus. It starts, but it throws an error when getting to text normalisation.

caffeinate mfa validate --clean --verbose --debug --config_path data/albanian_mfa_config.yaml --phone_set IPA data/albanian_mfa data/dictionary/albanian_mfa.dict
 INFO     Setting up corpus information...
 INFO     Loading corpus from source files...
 100% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 837/100  [ 0:00:01 < 0:00:00 , 4,176 it/s ]
 INFO     Found 80 speakers across 1252 files, average number of utterances per speaker: 15.65
 INFO     Initializing multiprocessing jobs...
 INFO     Normalizing text...
   0% ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1/1,252  [ 0:00:04 < -:--:-- , ? it/s ]
 ERROR    There was an error in the run, please see the log.
Error in atexit._run_exitfuncs:

For Reproducing your issue Please fill out the following:

  1. Corpus structure
    • What language is the corpus in? Albanian
    • How many files/speakers? 1252 files 80 speakers
    • Are you using lab files or TextGrid files for input? lab files
  2. Dictionary
    • Are you using a dictionary from MFA? If so, which one? No
    • If it's a custom dictionary, what is the phoneset? IPA
  3. Acoustic model
    • If you're using an acoustic model, is it one download through MFA? If so, which one? NA
    • If it's a model you've trained, what data was it trained on? NA

Log file

albanian_mfa.log

Desktop (please complete the following information):

Additional context

The conda environment

  aom                conda-forge/osx-arm64::aom-3.8.1-h078ce10_0
  atk-1.0            conda-forge/osx-arm64::atk-1.0-2.38.0-hcb7b3dd_1
  audioread          conda-forge/osx-arm64::audioread-3.0.1-py39h2804cbe_1
  baumwelch          conda-forge/osx-arm64::baumwelch-0.3.8-h2ffa867_0
  biopython          conda-forge/osx-arm64::biopython-1.79-py39h02fc5c5_3
  brotli             conda-forge/osx-arm64::brotli-1.1.0-hb547adb_1
  brotli-bin         conda-forge/osx-arm64::brotli-bin-1.1.0-hb547adb_1
  brotli-python      conda-forge/osx-arm64::brotli-python-1.1.0-py39hb198ff7_1
  bzip2              conda-forge/osx-arm64::bzip2-1.0.8-h93a5062_5
  ca-certificates    conda-forge/osx-arm64::ca-certificates-2024.2.2-hf0a4a13_0
  cairo              conda-forge/osx-arm64::cairo-1.18.0-hd1e100b_0
  certifi            conda-forge/noarch::certifi-2024.2.2-pyhd8ed1ab_0
  cffi               conda-forge/osx-arm64::cffi-1.16.0-py39he153c15_0
  charset-normalizer conda-forge/noarch::charset-normalizer-3.3.2-pyhd8ed1ab_0
  click              conda-forge/noarch::click-8.1.7-unix_pyh707e725_0
  colorama           conda-forge/noarch::colorama-0.4.6-pyhd8ed1ab_0
  contourpy          conda-forge/osx-arm64::contourpy-1.2.0-py39he9de807_0
  cycler             conda-forge/noarch::cycler-0.12.1-pyhd8ed1ab_0
  cython             conda-forge/osx-arm64::cython-3.0.8-py39hf3050f2_0
  dataclassy         conda-forge/noarch::dataclassy-1.0.1-pyhd8ed1ab_0
  dav1d              conda-forge/osx-arm64::dav1d-1.2.1-hb547adb_0
  decorator          conda-forge/noarch::decorator-5.1.1-pyhd8ed1ab_0
  expat              conda-forge/osx-arm64::expat-2.5.0-hb7217d7_1
  ffmpeg             conda-forge/osx-arm64::ffmpeg-6.1.1-gpl_h31ea89b_104
  font-ttf-dejavu-s~ conda-forge/noarch::font-ttf-dejavu-sans-mono-2.37-hab24e00_0
  font-ttf-inconsol~ conda-forge/noarch::font-ttf-inconsolata-3.000-h77eed37_0
  font-ttf-source-c~ conda-forge/noarch::font-ttf-source-code-pro-2.038-h77eed37_0
  font-ttf-ubuntu    conda-forge/noarch::font-ttf-ubuntu-0.83-h77eed37_1
  fontconfig         conda-forge/osx-arm64::fontconfig-2.14.2-h82840c6_0
  fonts-conda-ecosy~ conda-forge/noarch::fonts-conda-ecosystem-1-0
  fonts-conda-forge  conda-forge/noarch::fonts-conda-forge-1-0
  fonttools          conda-forge/osx-arm64::fonttools-4.49.0-py39h17cfd9d_0
  freetype           conda-forge/osx-arm64::freetype-2.12.1-hadb7bae_2
  fribidi            conda-forge/osx-arm64::fribidi-1.0.10-h27ca646_0
  gdk-pixbuf         conda-forge/osx-arm64::gdk-pixbuf-2.42.10-h15fa40c_4
  gettext            conda-forge/osx-arm64::gettext-0.21.1-h0186832_0
  giflib             conda-forge/osx-arm64::giflib-5.2.1-h1a8c8d9_3
  gmp                conda-forge/osx-arm64::gmp-6.3.0-h965bd2d_0
  gnutls             conda-forge/osx-arm64::gnutls-3.7.9-hd26332c_0
  graphite2          conda-forge/osx-arm64::graphite2-1.3.13-h9f76cd9_1001
  graphviz           conda-forge/osx-arm64::graphviz-9.0.0-h3face73_1
  greenlet           conda-forge/osx-arm64::greenlet-3.0.3-py39hf3050f2_0
  gtk2               conda-forge/osx-arm64::gtk2-2.24.33-h7895bb2_3
  gts                conda-forge/osx-arm64::gts-0.7.6-he42f4ea_4
  harfbuzz           conda-forge/osx-arm64::harfbuzz-8.3.0-h8f0ba13_0
  hdbscan            conda-forge/osx-arm64::hdbscan-0.8.33-py39h373d45f_4
  icu                conda-forge/osx-arm64::icu-73.2-hc8870d7_0
  idna               conda-forge/noarch::idna-3.6-pyhd8ed1ab_0
  importlib-resourc~ conda-forge/noarch::importlib-resources-6.1.1-pyhd8ed1ab_0
  importlib_resourc~ conda-forge/noarch::importlib_resources-6.1.1-pyhd8ed1ab_0
  joblib             conda-forge/noarch::joblib-1.3.2-pyhd8ed1ab_0
  kaldi              conda-forge/osx-arm64::kaldi-5.5.1112-cpu_h012d200_0
  kiwisolver         conda-forge/osx-arm64::kiwisolver-1.4.5-py39hbd775c9_1
  kneed              conda-forge/noarch::kneed-0.8.5-pyhd8ed1ab_0
  krb5               conda-forge/osx-arm64::krb5-1.21.2-h92f50d5_0
  lame               conda-forge/osx-arm64::lame-3.100-h1a8c8d9_1003
  lazy_loader        conda-forge/noarch::lazy_loader-0.3-pyhd8ed1ab_0
  lcms2              conda-forge/osx-arm64::lcms2-2.16-ha0e7c42_0
  lerc               conda-forge/osx-arm64::lerc-4.0.0-h9a09cb3_0
  libabseil          conda-forge/osx-arm64::libabseil-20230802.1-cxx17_h13dd4ca_0
  libass             conda-forge/osx-arm64::libass-0.17.1-hf7da4fe_1
  libblas            conda-forge/osx-arm64::libblas-3.9.0-21_osxarm64_openblas
  libbrotlicommon    conda-forge/osx-arm64::libbrotlicommon-1.1.0-hb547adb_1
  libbrotlidec       conda-forge/osx-arm64::libbrotlidec-1.1.0-hb547adb_1
  libbrotlienc       conda-forge/osx-arm64::libbrotlienc-1.1.0-hb547adb_1
  libcblas           conda-forge/osx-arm64::libcblas-3.9.0-21_osxarm64_openblas
  libcxx             conda-forge/osx-arm64::libcxx-16.0.6-h4653b0c_0
  libdeflate         conda-forge/osx-arm64::libdeflate-1.19-hb547adb_0
  libedit            conda-forge/osx-arm64::libedit-3.1.20191231-hc8eb9b7_2
  libexpat           conda-forge/osx-arm64::libexpat-2.5.0-hb7217d7_1
  libffi             conda-forge/osx-arm64::libffi-3.4.2-h3422bc3_5
  libflac            conda-forge/osx-arm64::libflac-1.4.3-hb765f3a_0
  libgd              conda-forge/osx-arm64::libgd-2.3.3-hfdf3952_9
  libgfortran        conda-forge/osx-arm64::libgfortran-5.0.0-13_2_0_hd922786_3
  libgfortran5       conda-forge/osx-arm64::libgfortran5-13.2.0-hf226fd6_3
  libglib            conda-forge/osx-arm64::libglib-2.78.4-h1635a5e_0
  libhwloc           conda-forge/osx-arm64::libhwloc-2.9.3-default_h4394839_1009
  libiconv           conda-forge/osx-arm64::libiconv-1.17-h0d3ecfb_2
  libidn2            conda-forge/osx-arm64::libidn2-2.3.7-h93a5062_0
  libjpeg-turbo      conda-forge/osx-arm64::libjpeg-turbo-3.0.0-hb547adb_1
  liblapack          conda-forge/osx-arm64::liblapack-3.9.0-21_osxarm64_openblas
  liblapacke         conda-forge/osx-arm64::liblapacke-3.9.0-21_osxarm64_openblas
  libllvm14          conda-forge/osx-arm64::libllvm14-14.0.6-hd1a9a77_4
  libogg             conda-forge/osx-arm64::libogg-1.3.4-h27ca646_1
  libopenblas        conda-forge/osx-arm64::libopenblas-0.3.26-openmp_h6c19121_0
  libopenvino        conda-forge/osx-arm64::libopenvino-2023.3.0-he6dadac_0
  libopenvino-arm-c~ conda-forge/osx-arm64::libopenvino-arm-cpu-plugin-2023.3.0-he6dadac_0
  libopenvino-auto-~ conda-forge/osx-arm64::libopenvino-auto-batch-plugin-2023.3.0-hc9f00d9_0
  libopenvino-auto-~ conda-forge/osx-arm64::libopenvino-auto-plugin-2023.3.0-hc9f00d9_0
  libopenvino-heter~ conda-forge/osx-arm64::libopenvino-hetero-plugin-2023.3.0-hf483cef_0
  libopenvino-ir-fr~ conda-forge/osx-arm64::libopenvino-ir-frontend-2023.3.0-hf483cef_0
  libopenvino-onnx-~ conda-forge/osx-arm64::libopenvino-onnx-frontend-2023.3.0-h98f6304_0
  libopenvino-paddl~ conda-forge/osx-arm64::libopenvino-paddle-frontend-2023.3.0-h98f6304_0
  libopenvino-pytor~ conda-forge/osx-arm64::libopenvino-pytorch-frontend-2023.3.0-hebf3989_0
  libopenvino-tenso~ conda-forge/osx-arm64::libopenvino-tensorflow-frontend-2023.3.0-hb5ee477_0
  libopenvino-tenso~ conda-forge/osx-arm64::libopenvino-tensorflow-lite-frontend-2023.3.0-hebf3989_0
  libopus            conda-forge/osx-arm64::libopus-1.3.1-h27ca646_1
  libpng             conda-forge/osx-arm64::libpng-1.6.42-h091b4b1_0
  libpq              conda-forge/osx-arm64::libpq-16.2-h0f8b458_0
  libprotobuf        conda-forge/osx-arm64::libprotobuf-4.25.1-h810fc01_1
  librosa            conda-forge/noarch::librosa-0.10.1-pyhd8ed1ab_0
  librsvg            conda-forge/osx-arm64::librsvg-2.56.3-h55a2576_1
  libsndfile         conda-forge/osx-arm64::libsndfile-1.2.2-h9739721_1
  libsqlite          conda-forge/osx-arm64::libsqlite-3.45.1-h091b4b1_0
  libtasn1           conda-forge/osx-arm64::libtasn1-4.19.0-h1a8c8d9_0
  libtiff            conda-forge/osx-arm64::libtiff-4.6.0-ha8a6c65_2
  libunistring       conda-forge/osx-arm64::libunistring-0.9.10-h3422bc3_0
  libvorbis          conda-forge/osx-arm64::libvorbis-1.3.7-h9f76cd9_0
  libvpx             conda-forge/osx-arm64::libvpx-1.13.1-hb765f3a_0
  libwebp            conda-forge/osx-arm64::libwebp-1.3.2-hf30222e_1
  libwebp-base       conda-forge/osx-arm64::libwebp-base-1.3.2-hb547adb_0
  libxcb             conda-forge/osx-arm64::libxcb-1.15-hf346824_0
  libxml2            conda-forge/osx-arm64::libxml2-2.12.5-h0d0cfa8_0
  libzlib            conda-forge/osx-arm64::libzlib-1.2.13-h53f4e23_5
  llvm-openmp        conda-forge/osx-arm64::llvm-openmp-17.0.6-hcd81f8e_0
  llvmlite           conda-forge/osx-arm64::llvmlite-0.42.0-py39h047a24b_1
  mad                conda-forge/osx-arm64::mad-0.15.1b-hbdafb3b_1
  markdown-it-py     conda-forge/noarch::markdown-it-py-3.0.0-pyhd8ed1ab_0
  matplotlib-base    conda-forge/osx-arm64::matplotlib-base-3.8.3-py39hbab7938_0
  mdurl              conda-forge/noarch::mdurl-0.1.2-pyhd8ed1ab_0
  montreal-forced-a~ conda-forge/noarch::montreal-forced-aligner-2.2.17-pyhd8ed1ab_0
  mpg123             conda-forge/osx-arm64::mpg123-1.32.4-hebf3989_0
  msgpack-python     conda-forge/osx-arm64::msgpack-python-1.0.7-py39he9de807_0
  munkres            conda-forge/noarch::munkres-1.1.4-pyh9f0ad1d_0
  ncurses            conda-forge/osx-arm64::ncurses-6.4-h463b476_2
  nettle             conda-forge/osx-arm64::nettle-3.9.1-h40ed0f5_0
  ngram              conda-forge/osx-arm64::ngram-1.3.15-h2ffa867_1
  numba              conda-forge/osx-arm64::numba-0.59.0-py39h278f47c_1
  numpy              conda-forge/osx-arm64::numpy-1.26.4-py39h7aa2656_0
  openfst            conda-forge/osx-arm64::openfst-1.8.3-h2ffa867_1
  openh264           conda-forge/osx-arm64::openh264-2.4.1-hebf3989_0
  openjpeg           conda-forge/osx-arm64::openjpeg-2.5.0-h4c1507b_3
  openssl            conda-forge/osx-arm64::openssl-3.2.1-h0d3ecfb_0
  p11-kit            conda-forge/osx-arm64::p11-kit-0.24.1-h29577a5_0
  packaging          conda-forge/noarch::packaging-23.2-pyhd8ed1ab_0
  pandas             conda-forge/osx-arm64::pandas-2.2.0-py39h47e51b9_0
  pango              conda-forge/osx-arm64::pango-1.50.14-hcf40dda_2
  patsy              conda-forge/noarch::patsy-0.5.6-pyhd8ed1ab_0
  pcre2              conda-forge/osx-arm64::pcre2-10.42-h26f9a81_0
  pgvector           conda-forge/osx-arm64::pgvector-0.6.0-h88c5ba3_0
  pgvector-python    conda-forge/noarch::pgvector-python-0.2.5-pyhe093146_0
  pillow             conda-forge/osx-arm64::pillow-10.2.0-py39h755f0b7_0
  pip                conda-forge/noarch::pip-24.0-pyhd8ed1ab_0
  pixman             conda-forge/osx-arm64::pixman-0.43.2-hebf3989_0
  platformdirs       conda-forge/noarch::platformdirs-4.2.0-pyhd8ed1ab_0
  pooch              conda-forge/noarch::pooch-1.8.0-pyhd8ed1ab_0
  postgresql         conda-forge/osx-arm64::postgresql-16.2-h1d0603d_0
  praatio            conda-forge/noarch::praatio-6.0.0-pyhd8ed1ab_0
  psycopg2           conda-forge/osx-arm64::psycopg2-2.9.9-py39h78886fb_0
  pthread-stubs      conda-forge/osx-arm64::pthread-stubs-0.4-h27ca646_1001
  pugixml            conda-forge/osx-arm64::pugixml-1.14-h13dd4ca_0
  pycparser          conda-forge/noarch::pycparser-2.21-pyhd8ed1ab_0
  pygments           conda-forge/noarch::pygments-2.17.2-pyhd8ed1ab_0
  pynini             conda-forge/osx-arm64::pynini-2.1.6-py39h48c5dd5_0
  pyparsing          conda-forge/noarch::pyparsing-3.1.1-pyhd8ed1ab_0
  pysocks            conda-forge/noarch::pysocks-1.7.1-pyha2e5f31_6
  pysoundfile        conda-forge/noarch::pysoundfile-0.12.1-pypyhd8ed1ab_1
  python             conda-forge/osx-arm64::python-3.9.18-hd7ebdb9_1_cpython
  python-dateutil    conda-forge/noarch::python-dateutil-2.8.2-pyhd8ed1ab_0
  python-tzdata      conda-forge/noarch::python-tzdata-2024.1-pyhd8ed1ab_0
  python_abi         conda-forge/osx-arm64::python_abi-3.9-4_cp39
  pytz               conda-forge/noarch::pytz-2024.1-pyhd8ed1ab_0
  pyyaml             conda-forge/osx-arm64::pyyaml-6.0.1-py39h0f82c59_1
  readline           conda-forge/osx-arm64::readline-8.2-h92ec313_1
  requests           conda-forge/noarch::requests-2.31.0-pyhd8ed1ab_0
  rich               conda-forge/noarch::rich-13.7.0-pyhd8ed1ab_0
  rich-click         conda-forge/noarch::rich-click-1.7.3-pyhd8ed1ab_0
  scikit-learn       conda-forge/osx-arm64::scikit-learn-1.2.2-py39hd5c4a62_2
  scipy              conda-forge/osx-arm64::scipy-1.12.0-py39hcc04109_2
  seaborn            conda-forge/noarch::seaborn-0.13.2-hd8ed1ab_0
  seaborn-base       conda-forge/noarch::seaborn-base-0.13.2-pyhd8ed1ab_0
  setuptools         conda-forge/noarch::setuptools-69.1.0-pyhd8ed1ab_0
  six                conda-forge/noarch::six-1.16.0-pyh6c4a22f_0
  snappy             conda-forge/osx-arm64::snappy-1.1.10-h17c5cce_0
  sox                conda-forge/osx-arm64::sox-14.4.2-h2353817_1018
  soxr               conda-forge/osx-arm64::soxr-0.1.3-h5008568_3
  soxr-python        conda-forge/osx-arm64::soxr-python-0.3.7-py39h373d45f_0
  sqlalchemy         conda-forge/osx-arm64::sqlalchemy-2.0.27-py39h17cfd9d_0
  sqlite             conda-forge/osx-arm64::sqlite-3.45.1-hf2abe2d_0
  statsmodels        conda-forge/osx-arm64::statsmodels-0.14.1-py39h373d45f_0
  svt-av1            conda-forge/osx-arm64::svt-av1-1.8.0-h463b476_0
  tbb                conda-forge/osx-arm64::tbb-2021.11.0-h2ffa867_1
  threadpoolctl      conda-forge/noarch::threadpoolctl-3.3.0-pyhc1e730c_0
  tk                 conda-forge/osx-arm64::tk-8.6.13-h5083fa2_1
  tqdm               conda-forge/noarch::tqdm-4.66.2-pyhd8ed1ab_0
  typing-extensions  conda-forge/noarch::typing-extensions-4.9.0-hd8ed1ab_0
  typing_extensions  conda-forge/noarch::typing_extensions-4.9.0-pyha770c72_0
  tzcode             conda-forge/osx-arm64::tzcode-2024a-h93a5062_0
  tzdata             conda-forge/noarch::tzdata-2024a-h0c530f3_0
  unicodedata2       conda-forge/osx-arm64::unicodedata2-15.1.0-py39h0f82c59_0
  urllib3            conda-forge/noarch::urllib3-2.2.1-pyhd8ed1ab_0
  wheel              conda-forge/noarch::wheel-0.42.0-pyhd8ed1ab_0
  x264               conda-forge/osx-arm64::x264-1!164.3095-h57fd34a_2
  x265               conda-forge/osx-arm64::x265-3.5-hbc6ce65_3
  xorg-libxau        conda-forge/osx-arm64::xorg-libxau-1.0.11-hb547adb_0
  xorg-libxdmcp      conda-forge/osx-arm64::xorg-libxdmcp-1.1.3-h27ca646_0
  xz                 conda-forge/osx-arm64::xz-5.2.6-h57fd34a_0
  yaml               conda-forge/osx-arm64::yaml-0.2.5-h3422bc3_2
  zipp               conda-forge/noarch::zipp-3.17.0-pyhd8ed1ab_0
  zlib               conda-forge/osx-arm64::zlib-1.2.13-h53f4e23_5
  zstd               conda-forge/osx-arm64::zstd-1.5.5-h4f39d0f_0
justinjhlo commented 7 months ago

Same error with Korean (using MFA's own dictionary) when I tried to downgrade from 3.0.0a8 to 2.2.17 to debug other errors downstream

stefanocoretta commented 7 months ago

I tried with 2.2.4 (which is the latest version I could find on conda-forge that has an arm64-specific binary for macOS arm64; not that it should matter perhaps?) and it works.

mmcauliffe commented 7 months ago

This should be fixed as of 3.0.0, just verify that your openfst is up to date for 1.8.3.