ModuleNotFoundError: No module named 'fairseq.models.speech_to_text.modules'

avidale commented 1 year ago

🐛 Bug

When importing anything from the main branch of Fairseq, an error occurs.

To Reproduce

Install fresh Fairseq

pip install 'git+https://github.com/facebookresearch/fairseq.git'

then try importing from it

 from fairseq.dataclass.configs import FairseqConfig

Result:

---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
[<ipython-input-2-d142f6ae2745>](https://localhost:8080/#) in <module>
----> 1 from fairseq.dataclass.configs import FairseqConfig

13 frames
[/usr/local/lib/python3.7/dist-packages/fairseq/__init__.py](https://localhost:8080/#) in <module>
     31 hydra_init()
     32 
---> 33 import fairseq.criterions  # noqa
     34 import fairseq.distributed  # noqa
     35 import fairseq.models  # noqa

[/usr/local/lib/python3.7/dist-packages/fairseq/criterions/__init__.py](https://localhost:8080/#) in <module>
     34     if file.endswith(".py") and not file.startswith("_"):
     35         file_name = file[: file.find(".py")]
---> 36         importlib.import_module("fairseq.criterions." + file_name)

[/usr/lib/python3.7/importlib/__init__.py](https://localhost:8080/#) in import_module(name, package)
    125                 break
    126             level += 1
--> 127     return _bootstrap._gcd_import(name[level:], package, level)
    128 
    129 

[/usr/local/lib/python3.7/dist-packages/fairseq/criterions/ctc.py](https://localhost:8080/#) in <module>
     19 from fairseq.dataclass import FairseqDataclass
     20 from fairseq.logging.meters import safe_round
---> 21 from fairseq.tasks import FairseqTask
     22 
     23 

[/usr/local/lib/python3.7/dist-packages/fairseq/tasks/__init__.py](https://localhost:8080/#) in <module>
    134 # automatically import any Python files in the tasks/ directory
    135 tasks_dir = os.path.dirname(__file__)
--> 136 import_tasks(tasks_dir, "fairseq.tasks")

[/usr/local/lib/python3.7/dist-packages/fairseq/tasks/__init__.py](https://localhost:8080/#) in import_tasks(tasks_dir, namespace)
    115         ):
    116             task_name = file[: file.find(".py")] if file.endswith(".py") else file
--> 117             importlib.import_module(namespace + "." + task_name)
    118 
    119             # expose `task_parser` for sphinx

[/usr/lib/python3.7/importlib/__init__.py](https://localhost:8080/#) in import_module(name, package)
    125                 break
    126             level += 1
--> 127     return _bootstrap._gcd_import(name[level:], package, level)
    128 
    129 

[/usr/local/lib/python3.7/dist-packages/fairseq/tasks/online_backtranslation.py](https://localhost:8080/#) in <module>
     32     encoders,
     33 )
---> 34 from fairseq.sequence_generator import SequenceGenerator
     35 from fairseq.tasks import register_task
     36 from fairseq.tasks.translation import TranslationTask, load_langpair_dataset

[/usr/local/lib/python3.7/dist-packages/fairseq/sequence_generator.py](https://localhost:8080/#) in <module>
     14 from fairseq import search, utils
     15 from fairseq.data import data_utils
---> 16 from fairseq.models import FairseqIncrementalDecoder
     17 from fairseq.ngram_repeat_block import NGramRepeatBlock
     18 

[/usr/local/lib/python3.7/dist-packages/fairseq/models/__init__.py](https://localhost:8080/#) in <module>
    233 # automatically import any Python files in the models/ directory
    234 models_dir = os.path.dirname(__file__)
--> 235 import_models(models_dir, "fairseq.models")

[/usr/local/lib/python3.7/dist-packages/fairseq/models/__init__.py](https://localhost:8080/#) in import_models(models_dir, namespace)
    215         ):
    216             model_name = file[: file.find(".py")] if file.endswith(".py") else file
--> 217             importlib.import_module(namespace + "." + model_name)
    218 
    219             # extra `model_parser` for sphinx

[/usr/lib/python3.7/importlib/__init__.py](https://localhost:8080/#) in import_module(name, package)
    125                 break
    126             level += 1
--> 127     return _bootstrap._gcd_import(name[level:], package, level)
    128 
    129 

[/usr/local/lib/python3.7/dist-packages/fairseq/models/speech_to_text/__init__.py](https://localhost:8080/#) in <module>
      5 
      6 from .berard import *  # noqa
----> 7 from .convtransformer import *  # noqa
      8 from .multi_modality_model import *  # noqa
      9 from .s2t_conformer import *  # noqa

[/usr/local/lib/python3.7/dist-packages/fairseq/models/speech_to_text/convtransformer.py](https://localhost:8080/#) in <module>
     21     register_model_architecture,
     22 )
---> 23 from fairseq.models.speech_to_text.modules.convolution import infer_conv_output_dim
     24 from fairseq.models.transformer import Embedding, TransformerDecoder
     25 from fairseq.modules import LayerNorm, PositionalEmbedding, TransformerEncoderLayer

ModuleNotFoundError: No module named 'fairseq.models.speech_to_text.modules'

---------------------------------------------------------------------------
NOTE: If your import is failing due to a missing package, you can
manually install dependencies using either !pip or !apt.

To view examples of installing some common dependencies, click the
"Open Examples" button below.
---------------------------------------------------------------------------

Expected behavior

The import succeeds.

Environment

fairseq Version (e.g., 1.0 or main): main
PyTorch Version (e.g., 1.0): does not matter
OS (e.g., Linux): does not matter
How you installed fairseq (pip, source): pip install 'git+https://github.com/facebookresearch/fairseq.git'
Build command you used (if compiling from source): NA
Python version: does not matter
CUDA/cuDNN version: NA
GPU models and configuration: NA

Additional context

NA

hirofumi0810 commented 1 year ago

@gwenzek Hi, thank you for reporting the issue. I tried to reproduce this, but I couldn't. Can you also confirm this?

avidale commented 1 year ago

Hi @hirofumi0810, please re-run this notebook, it does reproduce the issue. https://colab.research.google.com/drive/1erS7TRgZDzk1u8LTMdoBIvbqU2BOdo-c?usp=sharing

hirofumi0810 commented 1 year ago

@avidale Thank you for sharing the notebook with me. But I was able to run the notebook without any error...

gmryu commented 1 year ago

@avidale @hirofumi0810 I can confirm this error pops out. If anyone encounters it, it can be solved by adding fairseq to python path, like following:

import sys
sys.path.append("/usr/local/lib/python3.7/dist-packages/fairseq") # or where fairseq repository is kept

Note that the path where fairseq repository is kept may differ if you use different ways to download/install and different python version,etc.

So there was time I did not have to append the path. But now this appending becomes a must to me.

skulshreshtha commented 1 year ago

I realized that if instead of installing fairseq from the git repository, you do pip install fairseq, it runs without any errors

facebookresearch / fairseq