Error with posix_ipc semaphore when trying to run AutoPCP

[andysegura89]

Hello, I am trying to run AutoPCP on my computer but I get the following error. Any help would be greatly appreciated.

Error in call to target 'stopes.core.launcher.Launcher': AttributeError("'posix_ipc.Semaphore' object has no attribute 'value'") full_key: launcher

I ran all the following commands for installation:

python -m pip install --upgrade pip pip install wheel pip install fairseq==0.12.2 git clone https://github.com/facebookresearch/stopes.git cd stopes pip install -e '.[dev,mono,mining]' pip install -e '.[auto_pcp]'

I've tried creating a virtual environment for the project and installing all the dependencies on there but still get the same error. I am using an m1 Mac and I have version 1.1.1 of posix-ipc. Below is the command I am running along with the full stack trace.

`HYDRA_FULL_ERROR=1 python -m stopes.modules +compare_audios=AutoPCP_multilingual_v2 \ +compare_audios.input_file=input.tsv \ ++compare_audios.src_audio_column=src_audio \ ++compare_audios.tgt_audio_column=tgt_audio \ +compare_audios.named_columns=true \ +compare_audios.output_file=output.txt /Users/andy/miniconda3/lib/python3.11/site-packages/hydra/_internal/hydra.py:119: UserWarning: Future Hydra versions will no longer change working directory at job runtime by default. See https://hydra.cc/docs/1.2/upgrades/1.1_to_1.2/changes_to_job_working_dir/ for more information. ret = run_job( Error executing job with overrides: ['+compare_audios=AutoPCP_multilingual_v2', '+compare_audios.input_file=input.tsv', '++compare_audios.src_audio_column=src_audio', '++compare_audios.tgt_audio_column=tgt_audio', '+compare_audios.named_columns=true', '+compare_audios.output_file=output.txt'] Traceback (most recent call last): File "/Users/andy/miniconda3/lib/python3.11/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 92, in _call_target return target(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/andy/Desktop/UTEP/Fall23/Research/stopes/stopes/core/launcher.py", line 267, in init self.throttle = utils.AsyncIPCSemaphore( ^^^^^^^^^^^^^^^^^^^^^^^^ File "", line 8, in init File "/Users/andy/Desktop/UTEP/Fall23/Research/stopes/stopes/core/utils.py", line 678, in __post_init__ if self._sem and self._sem.value != self.initial_value: ^^^^^^^^^^^^^^^ AttributeError: 'posix_ipc.Semaphore' object has no attribute 'value'

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "", line 198, in _run_module_as_main File "", line 88, in _run_code File "/Users/andy/Desktop/UTEP/Fall23/Research/stopes/stopes/modules/main.py", line 63, in main() File "/Users/andy/miniconda3/lib/python3.11/site-packages/hydra/main.py", line 94, in decorated_main _run_hydra( File "/Users/andy/miniconda3/lib/python3.11/site-packages/hydra/_internal/utils.py", line 394, in _run_hydra _run_app( File "/Users/andy/miniconda3/lib/python3.11/site-packages/hydra/_internal/utils.py", line 457, in _run_app run_and_report( File "/Users/andy/miniconda3/lib/python3.11/site-packages/hydra/_internal/utils.py", line 223, in run_and_report raise ex File "/Users/andy/miniconda3/lib/python3.11/site-packages/hydra/_internal/utils.py", line 220, in run_and_report return func() ^^^^^^ File "/Users/andy/miniconda3/lib/python3.11/site-packages/hydra/_internal/utils.py", line 458, in lambda: hydra.run( ^^^^^^^^^^ File "/Users/andy/miniconda3/lib/python3.11/site-packages/hydra/internal/hydra.py", line 132, in run = ret.return_value ^^^^^^^^^^^^^^^^ File "/Users/andy/miniconda3/lib/python3.11/site-packages/hydra/core/utils.py", line 260, in return_value raise self._return_value File "/Users/andy/miniconda3/lib/python3.11/site-packages/hydra/core/utils.py", line 186, in run_job ret.return_value = task_function(task_cfg) ^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/andy/Desktop/UTEP/Fall23/Research/stopes/stopes/modules/main.py", line 43, in main launcher = hydra.utils.instantiate(config.launcher) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/andy/miniconda3/lib/python3.11/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 226, in instantiate return instantiate_node( ^^^^^^^^^^^^^^^^^ File "/Users/andy/miniconda3/lib/python3.11/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 347, in instantiate_node return _call_target(target, partial, args, kwargs, full_key) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/andy/miniconda3/lib/python3.11/site-packages/hydra/_internal/instantiate/_instantiate2.py", line 97, in _call_target raise InstantiationException(msg) from e hydra.errors.InstantiationException: Error in call to target 'stopes.core.launcher.Launcher': AttributeError("'posix_ipc.Semaphore' object has no attribute 'value'") full_key: launcher`

[avidale]

Hi @andysegura89! It looks like you are using a submitit launcher, which is expected to work with a Slurm cluster (this is what we are using by default in the FAIR internal cluster). Please try adding the launcher=local line to your command; it will use a local launcher without the semaphore that is apparently causing the problem.

[avidale]

In my experience, when running the commend locally, it might be also needed to export the USER environment variable, to please Hydra. So the full command could look like

export USER=anything && python -m stopes.modules +compare_audios=AutoPCP_multilingual_v2 \
    +compare_audios.input_file=input.tsv \
    compare_audios.src_audio_column=src_audio \
    compare_audios.tgt_audio_column=tgt_audio \ 
    +compare_audios.named_columns=true \
    +compare_audios.output_file=output.txt \ 
    launcher=local