MontrealCorpusTools / Montreal-Forced-Aligner

Command line utility for forced alignment using Kaldi
https://montrealcorpustools.github.io/Montreal-Forced-Aligner/
MIT License
1.3k stars 243 forks source link

[BUG] Type annotation for "Word.dictionary" can't be correctly interpreted for Annotated Declarative Table form. #551

Open timspeech opened 1 year ago

timspeech commented 1 year ago

Debugging checklist

[√] Have you updated to latest MFA version? [√ ] Have you tried rerunning the command with the --clean flag?

Describe the issue A clear and concise description of what the bug is.

using mfa version 2.1.2, and running mfa model download dictionary english_us_arpa the returning is sqlalchemy.exc.ArgumentError: Type annotation for "Word.dictionary" can't be correctly interpreted for Annotated Declarative Table form. ORM annotations should normally make use of the Mapped[] generic type, or other ORM-compatible generic type, as a container for the actual type, which indicates the intent that the attribute is mapped. Class variables that are not intended to be mapped by the ORM should use ClassVar[]. To allow Annotated Declarative to disregard legacy annotations which don't use Mapped[] to pass, set "__allow_unmapped__ = True" on the class or a superclass this class. (Background on this error at: https://sqlalche.me/e/20/zlpr)

For Reproducing your issue Please fill out the following:

  1. Corpus structure
    • What language is the corpus in?
    • How many files/speakers?
    • Are you using lab files or TextGrid files for input?
  2. Dictionary
    • Are you using a dictionary from MFA? If so, which one?
    • If it's a custom dictionary, what is the phoneset?
  3. Acoustic model
    • If you're using an acoustic model, is it one download through MFA? If so, which one?
    • If it's a model you've trained, what data was it trained on?

Log file Please attach the log file for the run that encountered an error (by default these will be stored in ~/Documents/MFA).

Desktop (please complete the following information):

Additional context Add any other context about the problem here.

mmcauliffe commented 1 year ago

I'm not sure what's generating this, since the 2,1.2 code doesn't have the type annotation it's complaining about (https://github.com/MontrealCorpusTools/Montreal-Forced-Aligner/blob/main/montreal_forced_aligner/db.py#L559). Could you try making a fresh conda environment with mfa and trying again? It might be a pycache issue.

timspeech commented 1 year ago

Thanks! I make a fresh conda environment with mfa and try again. This time, when runing mfa model inspect dictionary english_us_arpa I got the following error ############################################################ sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server: Connection refused Is the server running on host "localhost" (::1) and accepting TCP/IP connections on port 5433? could not connect to server: Connection refused Is the server running on host "localhost" (127.0.0.1) and accepting TCP/IP connections on port 5433? ############################################################

And inside pg_init_log_global.txt, it shows ############################################################ initdb: cannot be run as root Please log in (using, e.g., "su") as the (unprivileged) user that will own the server process. ############################################################ I've tried to su, and then run mfa command, but the pg_init_log_global.txt refreshs and still shows the above.

mmcauliffe commented 1 year ago

What's the exact command you're trying to run? It shoudn't require sudo or anything having to do with root. Can you try also running mfa configure --database_port 5433? There might be a conflict with a system install of postgres (2.1.2 has database_port set to 5433 by default, but an earlier version would have set it to 5432).

timspeech commented 1 year ago

Thanks for your reply!

The exact command I run is mfa model inspect dictionary english_us_arpa

I've runed mfa configure --database_port 5433, and the same error occurs, listed as below

sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) could not connect to server: Connection refused
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5433?
could not connect to server: Connection refused
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5433?

Further more, if I run

mfa align
    raw_data/CORPUS \
    mandarin_mfa \
    mandarin_mfa \
    preprocessed_data/CORPUS

(I've downloaded the mandarin_mfa acoustic models and mandarin_mfa dictionary. ) It will return the following error:

subprocess.CalledProcessError: Command '['initdb', '-D', '/root/Documents/MFA/pg_mfa_global', '--encoding=UTF8']' returned non-zero exit status 1.

and the related error is

initdb: cannot be run as root
Please log in (using, e.g., "su") as the (unprivileged) user that will
own the server process.
mmcauliffe commented 1 year ago

Are you logged into the root user? Or is this related to docker? Can you log in as a non-root user?

Yymax-max commented 1 year ago

i also meet this problem.how to solve it?

Yymax-max commented 1 year ago

@mmcauliffe i also meet this problem.how to solve it?

mmcauliffe commented 1 year ago

@mmcauliffe i also meet this problem.how to solve it?

What are the specifications of your machine? How are you trying to run MFA?

Desktop (please complete the following information):

Are you logged into the root user? Can you try logging into a different account?

timspeech commented 1 year ago

Are you logged into the root user? Or is this related to docker? Can you log in as a non-root user?

@mmcauliffe Hi, I'm logged as the root user, and I can't log as a non-root user. This is run on CENTOS linux, with no docker.

mmcauliffe commented 1 year ago

Ok, can you create a new non-root user via: https://www.cyberciti.biz/faq/create-a-new-user-account-in-centos-7-8-linux/, and then use that account to run MFA?

Yymax-max commented 1 year ago

@mmcauliffe我也遇到这个问题,怎么解决?

你的机器规格是多少?您打算如何运行 MFA?

桌面(请填写以下信息):

  • 操作系统:[例如 Windows、OSX、Linux]
  • 版本 [例如 MacOSX 10.15、Ubuntu 20.04、Windows 10 等]
  • 有关设置的任何其他详细信息(云、Docker 等)

您是否已登录到 root 用户?您可以尝试登录其他帐户吗?

i run on docker and mfa version is 2.0.6. I am not update to the latest version. I am using "conda env create -n mfa-dev -f environment.yml" and "pip install -e .[dev]" to Source installation. Installation is successful. But when use mfa align is error like "sqlalchemy.exc.ArgumentError: Type annotation for "Word.dictionary" can't be correctly interpreted for Annotated Declarative Table form. ORM annotations should normally make use of the Mapped[] generic type, or other ORM-compatible generic type, as a container for the actual type, which indicates the intent that the attribute is mapped. Class variables that are not intended to be mapped by the ORM should use ClassVar[]. To allow Annotated Declarative to disregard legacy annotations which don't use Mapped[] to pass, set "allow_unmapped = True" on the class or a superclass this class. (Background on this error at: https://sqlalche.me/e/20/zlpr)"

mmcauliffe commented 1 year ago

@Yymax-max For 2.0.6 modify the environment.yml to have sqlalchemy<2.0 and praatio<6.0 and it should work.

Yymax-max commented 1 year ago

@Yymax-max For 2.0.6 modify the environment.yml to have sqlalchemy<2.0 and praatio<6.0 and it should work.

it's working! Thank you very much for your work, especially useful!mfa is getting better and better!

timspeech commented 1 year ago

Ok, can you create a new non-root user via: https://www.cyberciti.biz/faq/create-a-new-user-account-in-centos-7-8-linux/, and then use that account to run MFA?

@mmcauliffe Sorry due to some constriction, I'm not allowed to login as non-root user, is there other alternative to tackle the issue? Thanks.

Yymax-max commented 1 year ago

@Yymax-max For 2.0.6 modify the environment.yml to have sqlalchemy<2.0 and praatio<6.0 and it should work. hi @mmcauliffe .When i use "mfa align ...." ,in the step of output Textgrid ,meeting the problem like this .

File "/opt/conda/envs/mfa-dev/lib/python3.10/multiprocessing/process.py", line 314, in _bootstrap self.run() File "/code/py_code/align/Montreal-Forced-Aligner/montreal_forced_aligner/textgrid.py", line 217, in export_textgrid if tier_name not in tg.tierNameList: File "/code/py_code/align/Montreal-Forced-Aligner/montreal_forced_aligner/alignment/multiprocessing.py", line 1180, in run export_textgrid( File "/code/py_code/align/Montreal-Forced-Aligner/montreal_forced_aligner/textgrid.py", line 217, in export_textgrid if tier_name not in tg.tierNameList: AttributeError: 'Textgrid' object has no attribute 'tierNameList' AttributeError: 'Textgrid' object has no attribute 'tierNameList'

if any question in environment.yml ?

mmcauliffe commented 1 year ago

make sure that you have praatio=5.1.1 installed in the environment. MFA 2.1+ works with praatio 6.0, but 2.0.6 needs 5.1.1.

mmcauliffe commented 1 year ago

Ok, can you create a new non-root user via: https://www.cyberciti.biz/faq/create-a-new-user-account-in-centos-7-8-linux/, and then use that account to run MFA?

@mmcauliffe Sorry due to some constriction, I'm not allowed to login as non-root user, is there other alternative to tackle the issue? Thanks.

@timspeech Is there anything restriction on creating users? Once you're in root, you should be able to log in as the created user from the root shell (and this would be how I would think about solving it). Another alternative would be to set up a system PostgreSQL installation, and then use mfa configure --database_port 5432 to point it, but I'm not too sure the exact steps you'd need to do, and whether you'd again need to create a postgres user on the machine rather than root to run the PostgreSQL dictionary.

If setting up Postgres on that machine is a nonstarter, you might have to just install MFA 2.0.6 for the moment via conda create -n aligner montreal-forced-aligner=2.0.6 praatio=5.1.1 sqlachemy=1.4.3. Nothing I've found online has any other solutions for the root issue other than creating a new user and using that to run postgres.

SolomidHero commented 1 year ago

@mmcauliffe I guess your command has typos and should look like this

conda create -n aligner -c conda-forge montreal-forced-aligner=2.0.6 praatio=5.1.1 sqlalchemy=1.4.3