search

facebookresearch / fairseq2

FAIR Sequence Modeling Toolkit 2
https://facebookresearch.github.io/fairseq2/
MIT License
667 stars 76 forks source link

OSError: libsndfile is not found!. Use your system package manager to install it (e.g. `apt install libsndfile1` #10

Closed mdotvasylenchuk closed 12 months ago

mdotvasylenchuk commented 1 year ago

Hi, I have a bug when try to run https://github.com/facebookresearch/seamless_communication repo: $ m4t_predict

Traceback (most recent call last):
  File "/home/myuser/.local/bin/m4t_predict", line 5, in <module>
    from m4t_scripts.predict.predict import main
  File "/home/myuser/.local/lib/python3.8/site-packages/m4t_scripts/predict/predict.py", line 10, in <module>
    from seamless_communication.models.inference import Translator
  File "/home/myuser/.local/lib/python3.8/site-packages/seamless_communication/models/inference/__init__.py", line 6, in <module>
    from seamless_communication.models.inference.translator import Translator as Translator
  File "/home/myuser/.local/lib/python3.8/site-packages/seamless_communication/models/inference/translator.py", line 12, in <module>
    from fairseq2.data import Collater
  File "/home/myuser/.local/lib/python3.8/site-packages/fairseq2/data/__init__.py", line 7, in <module>
    from fairseq2.data.cstring import CString as CString
  File "/home/myuser/.local/lib/python3.8/site-packages/fairseq2/data/cstring.py", line 58, in <module>
    from fairseq2n.bindings.data.string import CString as CString
  File "/home/myuser/.local/lib/python3.8/site-packages/fairseq2n/__init__.py", line 91, in <module>
    _load_sndfile()
  File "/home/myuser/.local/lib/python3.8/site-packages/fairseq2n/__init__.py", line 81, in _load_sndfile
    raise OSError(
OSError: libsndfile is not found!. Use your system package manager to install it (e.g. `apt install libsndfile1`).

SeamlessM4T installed by instruction in README.md by pip install .

Installation verification of libsndfile1

$ dpkg -L libsndfile1
/.
/usr
/usr/lib
/usr/lib/x86_64-linux-gnu
/usr/lib/x86_64-linux-gnu/libsndfile.so.1.0.28
/usr/share
/usr/share/doc
/usr/share/doc/libsndfile1
/usr/share/doc/libsndfile1/changelog.Debian.gz
/usr/share/doc/libsndfile1/copyright
/usr/lib/x86_64-linux-gnu/libsndfile.so.1

System info: Linux DD8SXD3 5.15.0-76-generic #83~20.04.1-Ubuntu SMP Wed Jun 21 20:23:31 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux memory 31GiB System memory processor Intel(R) Core(TM) i7-10850H CPU @ 2.70GHz bridge 10th Gen Core Processor Host Bridge/DRAM Registers display UHD Graphics

cbalioglu commented 1 year ago

@mdotvasylenchuk I see that you installed libsndfile using apt. Do you run m4t_predict within a Conda environment?

mdotvasylenchuk commented 1 year ago

@cbalioglu yes, here is full trace

(base)$ conda install -y -c conda-forge libsndfile
Retrieving notices: ...working... DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): conda.anaconda.org:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/r/notices.json HTTP/1.1" 404 None
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/notices.json HTTP/1.1" 404 None
DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/notices.json HTTP/1.1" 404 None
done
Collecting package metadata (current_repodata.json): - DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): conda.anaconda.org:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): conda.anaconda.org:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443
| DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/noarch/current_repodata.json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/linux-64/current_repodata.json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/r/noarch/current_repodata.json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/r/linux-64/current_repodata.json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/linux-64/current_repodata.json HTTP/1.1" 200 None
DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/noarch/current_repodata.json HTTP/1.1" 200 None
done
Solving environment: done

## Package Plan ##

  environment location: /home/myuser/anaconda3

  added / updated specs:
    - libsndfile

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    ca-certificates-2023.7.22  |       hbcca054_0         146 KB  conda-forge
    certifi-2023.7.22          |     pyhd8ed1ab_0         150 KB  conda-forge
    conda-23.7.3               |  py311h38be061_0         1.3 MB  conda-forge
    cryptography-39.0.1        |  py311h9ce1e76_0         1.4 MB
    gettext-0.21.0             |       hf68c758_0         2.6 MB
    libflac-1.3.4              |       h27087fc_0         474 KB  conda-forge
    libogg-1.3.4               |       h7f98852_1         206 KB  conda-forge
    libopus-1.3.1              |       h7f98852_1         255 KB  conda-forge
    libsndfile-1.0.31          |       h9c3ff4c_1         602 KB  conda-forge
    libvorbis-1.3.7            |       h9c3ff4c_0         280 KB  conda-forge
    openssl-1.1.1v             |       h7f8727e_0         3.7 MB
    python_abi-3.11            |          2_cp311           5 KB  conda-forge
    ------------------------------------------------------------
                                           Total:        11.1 MB

The following NEW packages will be INSTALLED:

  gettext            pkgs/main/linux-64::gettext-0.21.0-hf68c758_0 
  libflac            conda-forge/linux-64::libflac-1.3.4-h27087fc_0 
  libogg             conda-forge/linux-64::libogg-1.3.4-h7f98852_1 
  libopus            conda-forge/linux-64::libopus-1.3.1-h7f98852_1 
  libsndfile         conda-forge/linux-64::libsndfile-1.0.31-h9c3ff4c_1 
  libvorbis          conda-forge/linux-64::libvorbis-1.3.7-h9c3ff4c_0 
  python_abi         conda-forge/linux-64::python_abi-3.11-2_cp311 

The following packages will be UPDATED:

  ca-certificates    pkgs/main::ca-certificates-2023.05.30~ --> conda-forge::ca-certificates-2023.7.22-hbcca054_0 
  conda              pkgs/main::conda-23.7.2-py311h06a4308~ --> conda-forge::conda-23.7.3-py311h38be061_0 
  openssl                                 1.1.1u-h7f8727e_0 --> 1.1.1v-h7f8727e_0 

The following packages will be SUPERSEDED by a higher-priority channel:

  certifi            pkgs/main/linux-64::certifi-2023.7.22~ --> conda-forge/noarch::certifi-2023.7.22-pyhd8ed1ab_0 

The following packages will be DOWNGRADED:

  cryptography                       41.0.2-py311h774aba0_0 --> 39.0.1-py311h9ce1e76_0 

Downloading and Extracting Packages
cryptography-39.0.1  | 1.4 MB    |                                                                                                                                                |   0%DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443                                                                                              |   0% 
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443                                                                                             |   0% 

                                                                                                                                                                                        DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): conda.anaconda.org:443
python_abi-3.11      | 5 KB      |                                                                                                                                                |   0% 
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): conda.anaconda.org:443
openssl-1.1.1v       | 3.7 MB    |                                                                                                                                                |   0%DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/linux-64/openssl-1.1.1v-h7f8727e_0.conda HTTP/1.1" 200 3914510                                          |   0% 
DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/linux-64/cryptography-39.0.1-py311h9ce1e76_0.conda HTTP/1.1" 200 1427919
cryptography-39.0.1  | 1.4 MB    | #6                                                                                                                                             |   1% DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/linux-64/python_abi-3.11-2_cp311.tar.bz2 HTTP/1.1" 200 4621                                         |   0% 
DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): repo.anaconda.com:443                                                                                                   DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/linux-64/libsndfile-1.0.31-h9c3ff4c_1.tar.bz2 HTTP/1.1" 200 615959
cryptography-39.0.1  | 1.4 MB    | ###########################6                                                                                                                   |  20% DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/linux-64/ca-certificates-2023.7.22-hbcca054_0.conda HTTP/1.1" 200 149515########################### | 100% 
libsndfile-1.0.31    | 602 KB    | ###7                                                                                                                                           |   3%DEBUG:urllib3.connectionpool:https://repo.anaconda.com:443 "GET /pkgs/main/linux-64/gettext-0.21.0-hf68c758_0.conda HTTP/1.1" 200 2769333                                          |   0% 
cryptography-39.0.1  | 1.4 MB    | ############################################################2                                                                                  |  42%DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): conda.anaconda.org:443                                                                                             |   1% 
DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/linux-64/conda-23.7.3-py311h38be061_0.conda HTTP/1.1" 200 1342708#####3                             |  80% 
ca-certificates-2023 | 146 KB    | ###############5                                                                                                                               |  11%DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): conda.anaconda.org:443                                                                                             |   4% 
cryptography-39.0.1  | 1.4 MB    | ############################################################################################################################################## | 100%DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/noarch/certifi-2023.7.22-pyhd8ed1ab_0.conda HTTP/1.1" 200 153791                                     |  36% 
openssl-1.1.1v       | 3.7 MB    | #########################################################################1                                                                     |  51%DEBUG:urllib3.connectionpool:Starting new HTTPS connection (1): conda.anaconda.org:443
DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/linux-64/libogg-1.3.4-h7f98852_1.tar.bz2 HTTP/1.1" 200 210550###################################### | 100% 
                                                                                                                                                                                        DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/linux-64/libvorbis-1.3.7-h9c3ff4c_0.tar.bz2 HTTP/1.1" 200 286280######6                              |  79% 
                                                                                                                                                                                        DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/linux-64/libflac-1.3.4-h27087fc_0.tar.bz2 HTTP/1.1" 200 484942
certifi-2023.7.22    | 150 KB    | ###############1                                                                                                                               |  11%DEBUG:urllib3.connectionpool:https://conda.anaconda.org:443 "GET /conda-forge/linux-64/libopus-1.3.1-h7f98852_1.tar.bz2 HTTP/1.1" 200 260658

Preparing transaction: done                                                                                                                                                              
Verifying transaction: done                                                                                                                                                              
Executing transaction: done                           

(base) $ m4t_predict 'take it' t2tt ukrainian --src_lang english
Traceback (most recent call last):                                                                                                                                                       
  File "/home/myuser/.local/bin/m4t_predict", line 5, in <module>                                                                                                                       
    from m4t_scripts.predict.predict import main                                                                                                                                         
  File "/home/myuser/.local/lib/python3.8/site-packages/m4t_scripts/predict/predict.py", line 10, in <module>                                                                           
    from seamless_communication.models.inference import Translator                                                                                                                       
  File "/home/myuser/.local/lib/python3.8/site-packages/seamless_communication/models/inference/__init__.py", line 6, in <module>                                                       
    from seamless_communication.models.inference.translator import Translator as Translator                                                                                              
  File "/home/myuser/.local/lib/python3.8/site-packages/seamless_communication/models/inference/translator.py", line 12, in <module>
    from fairseq2.data import Collater
  File "/home/myuser/.local/lib/python3.8/site-packages/fairseq2/data/__init__.py", line 7, in <module>
    from fairseq2.data.cstring import CString as CString
  File "/home/myuser/.local/lib/python3.8/site-packages/fairseq2/data/cstring.py", line 58, in <module>
    from fairseq2n.bindings.data.string import CString as CString
  File "/home/myuser/.local/lib/python3.8/site-packages/fairseq2n/__init__.py", line 91, in <module>
    _load_sndfile()
  File "/home/myuser/.local/lib/python3.8/site-packages/fairseq2n/__init__.py", line 81, in _load_sndfile
    raise OSError(
OSError: libsndfile is not found!. Use your system package manager to install it (e.g. `apt install libsndfile1`).
TTTdas commented 1 year ago

@cbalioglu The same problem occurred when I installed libsndfile using conda

cbalioglu commented 1 year ago

@mdotvasylenchuk @TTTdas I just released v0.1.1 with updated installation instructions in README. Could you please give it another try in a clean environment? Thanks!

cocktailpeanut commented 1 year ago

@cbalioglu I'm getting this error as well with the v0.1.1 on a Mac.

Here's what I did (I'm using this in seamless_communication)

  1. I installed libsndfile with brew install libsndfile
  2. Then I installed the requirements in seamless_communication
  3. I checked to confirm that fairseq2 is 0.1.1

When I run seamless_communication I get this error.

I also tried different approaches such as those mentioned here https://github.com/ohmtech-rdi/eurorack-blocks/issues/444#issuecomment-1153155080 but can't get it to work.

My machine is Mac M1 Max. Were you able to make it work on your Mac?

cbalioglu commented 12 months ago

@cocktailpeanut What is the specific error message you are receiving with v0.1.1? Are you %100 sure that you are not using a Conda environment (not even "base")? Is Homebrew installed in its default location (i.e. /opt/homebrew on Mac M1)?

TTTdas commented 12 months ago

@cbalioglu Thanks! It works! Just after I disabled conda completely. It seems that conda environment affected the lib path (even if I changed it)?

cbalioglu commented 12 months ago

Happy to hear that! Closing this issue, but feel free to re-open it (or open a new one) if you face a similar issue in the future.

zapfruit commented 11 months ago

@cbalioglu Thanks! It works! Just after I disabled conda completely. It seems that conda environment affected the lib path (even if I changed it)?

hi I am also facing the same issue can you pls elaborate on what you did exactly, it seems i still can't fix the error after disabling the conda env getting this error: UserWarning: CUDA initialization: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero. (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.) return torch._C._cuda_getDeviceCount() > 0 2023-09-19 12:04:44,242 INFO -- m4t_scripts.predict.predict: Running inference on the CPU in torch.float32. Using the cached checkpoint of the model 'seamlessM4T_medium'. Set force=True to download again. Using the cached tokenizer of the model 'seamlessM4T_medium'. Set force=True to download again. Using the cached checkpoint of the model 'vocoder_36langs'. Set force=True to download again. ValueError: File contains data in an unknown format.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):File "/home/User/.local/bin/m4t_predict", line 8, in sys.exit(main()) File "/home/User/.local/lib/python3.8/site-packages/m4t_scripts/predict/predict.py", line 72, in main translated_text, wav, sr = translator.predict( File "/home/User/.local/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/home/User/.local/lib/python3.8/site-packages/seamless_communication/models/inference/translator.py", line 207, in predict decoded_audio = self.decode_audio(block) ValueError: The input audio cannot be decoded. See nested exception for details.

TTTdas commented 11 months ago

@zapfruit

@cbalioglu Thanks! It works! Just after I disabled conda completely. It seems that conda environment affected the lib path (even if I changed it)?

hi I am also facing the same issue can you pls elaborate on what you did exactly, it seems i still can't fix the error after disabling the conda env getting this error: UserWarning: CUDA initialization: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero. (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.) return torch._C._cuda_getDeviceCount() > 0 2023-09-19 12:04:44,242 INFO -- m4t_scripts.predict.predict: Running inference on the CPU in torch.float32. Using the cached checkpoint of the model 'seamlessM4T_medium'. Set force=True to download again. Using the cached tokenizer of the model 'seamlessM4T_medium'. Set force=True to download again. Using the cached checkpoint of the model 'vocoder_36langs'. Set force=True to download again. ValueError: File contains data in an unknown format.

The above exception was the direct cause of the following exception:

Traceback (most recent call last):File "/home/User/.local/bin/m4t_predict", line 8, in sys.exit(main()) File "/home/User/.local/lib/python3.8/site-packages/m4t_scripts/predict/predict.py", line 72, in main translated_text, wav, sr = translator.predict( File "/home/User/.local/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/home/User/.local/lib/python3.8/site-packages/seamless_communication/models/inference/translator.py", line 207, in predict decoded_audio = self.decode_audio(block) ValueError: The input audio cannot be decoded. See nested exception for details.

It seems to be a problem on CUDA. Do you have the cuda vesion that matches the torch version? When the conda environment is disabled, other dependencies need to be installed properly as well (i.e. cuda, cudnn).

zapfruit commented 11 months ago

@zapfruit

@cbalioglu Thanks! It works! Just after I disabled conda completely. It seems that conda environment affected the lib path (even if I changed it)?

hi I am also facing the same issue can you pls elaborate on what you did exactly, it seems i still can't fix the error after disabling the conda env getting this error: UserWarning: CUDA initialization: CUDA unknown error - this may be due to an incorrectly set up environment, e.g. changing env variable CUDA_VISIBLE_DEVICES after program start. Setting the available devices to be zero. (Triggered internally at ../c10/cuda/CUDAFunctions.cpp:109.) return torch._C._cuda_getDeviceCount() > 0 2023-09-19 12:04:44,242 INFO -- m4t_scripts.predict.predict: Running inference on the CPU in torch.float32. Using the cached checkpoint of the model 'seamlessM4T_medium'. Set force=True to download again. Using the cached tokenizer of the model 'seamlessM4T_medium'. Set force=True to download again. Using the cached checkpoint of the model 'vocoder_36langs'. Set force=True to download again. ValueError: File contains data in an unknown format. The above exception was the direct cause of the following exception: Traceback (most recent call last):File "/home/User/.local/bin/m4t_predict", line 8, in sys.exit(main()) File "/home/User/.local/lib/python3.8/site-packages/m4t_scripts/predict/predict.py", line 72, in main translated_text, wav, sr = translator.predict( File "/home/User/.local/lib/python3.8/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) File "/home/User/.local/lib/python3.8/site-packages/seamless_communication/models/inference/translator.py", line 207, in predict decoded_audio = self.decode_audio(block) ValueError: The input audio cannot be decoded. See nested exception for details.

It seems to be a problem on CUDA. Do you have the cuda vesion that matches the torch version? When the conda environment is disabled, other dependencies need to be installed properly as well (i.e. cuda, cudnn).

it wasnt a problem with cuda just the wrong file format, seems it works better for .wav files i was using a .mp3 one. But its working now. Thanks a lot!

OpenSourceCommunityInterface commented 2 months ago

Mac M1 Max ,as the same