Huggingface transformers inference: ModuleNotFoundError: No module named 'generate'

[steve-marmalade]

When running the imports listed in transformers.md:

from PIL import Image
from torchvision import transforms
from transformers import OFATokenizer, OFAModel
from generate import sequence_generator

I get ModuleNotFoundError: No module named 'generate'

Where is generate supposed to come from? The implementations of sequence_generator.SequenceGenerator that I see in e.g. fairseq also don't have the same signature, so it's not clear how to proceed.

[JustinLin610]

Did you clone the latest code of the branch feature/add_transformers? Cuz in the new code we have added a directory called generate with sequence_generator inside.

[steve-marmalade]

I'll be really embarrassed if I've made a silly mistake here, but I don't see a directory called generate in that branch:

[faychu]

Here: https://github.com/OFA-Sys/OFA/tree/feature/add_transformers/transformers/src/transformers/models/ofa/generate

[steve-marmalade]

Excellent, thank you both for the quick help. I never would have found it otherwise.

I have gotten the import to work by adding /path/to/OFA/transformers/src/transformers/models/ofa to my python path.

Perhaps we should either:

• add a sentence to transformers.md mentioning to add this directory to the python path
• or add an __init__.py in the generate/ directory and update the import to be from transformers.models.ofa.generate import sequence_generator

What do you think?

[JustinLin610]

Good suggestion! Thanks for helping us improve our repo. I'll update the information to make things clear!

[Sultanax]

Hi! I'm still having this issue, even though I've downloaded the repo using git clone --single-branch --branch feature/add_transformers https://github.com/OFA-Sys/OFA.git and also have transformers/src/transformers/models/ofa/generate. I've followed the steps listed in transformers.md, but am still receiving the error:

    from generate import sequence_generator
ModuleNotFoundError: No module named 'generate'

@Steve-marmalade Could you share the steps you took to solve this issue?

[steve-marmalade]

Hi @Sultanax, sure. I did two things:

1. Added an empty __init__.py file in transformers/src/transformers/models/ofa/generate/__init__.py
2. Changed the import to be from transformers.models.ofa.generate import sequence_generator

Note: you will need to pip install OFA/transformers/ again

[Sultanax]

Thanks for the suggestions! I've completed those steps, but using from transformers.models.ofa.generate import sequence_generator still generates the same error:

    from transformers.models.ofa.generate import sequence_generator
ModuleNotFoundError: No module named 'transformers.models.ofa.generate'

I think the problem I'm mainly having is that when I run pip install OFA/transformers/ in the command line, I get the following error:

Hint: It looks like a path. File 'OFA/transformers/' does not exist.

I already have Huggingface transformers installed (version 4.21.1). I'm not sure exactly how to work around this. @JustinLin610

[JustinLin610]

Sorry for the problem pretty much. If you would like to directly import generate, you should cd to the directory of modeling_ofa.py. A better solution should be the previously mentioned one with __init__.py and reinstallation.

Seemingly your installation is not successful. How about cd into transformers with a setup.py, and uninstall your old transformers and then run pip install . to install ours?

[JustinLin610]

Hi @Sultanax, sure. I did two things:

1. Added an empty __init__.py file in transformers/src/transformers/models/ofa/generate/__init__.py
2. Changed the import to be from transformers.models.ofa.generate import sequence_generator

Note: you will need to pip install OFA/transformers/ again

I have added the empty __init__.py to the directory generate and it all works now. If you guys are interested, you can try this notebook (link) and see if there are still any problems.