No module named 'lib.xrenner'

[roei-shlezinger]

I got this error:

File "C:\Users\user\Desktop\temp\HebPipe\hebpipe\heb_pipe.py", line 25, in from lib.xrenner import Xrenner ModuleNotFoundError: No module named 'lib.xrenner'

After installing xrenner, the problem persists. Could you recommend a course of action?

@amir-zeldes

[amir-zeldes]

Can you give more details on how you installed the system? Did you use pip or maybe setup.py? It looks like it's in a temp directory so I'm guessing that's not the case?

The recommended way to install the system with all dependencies, which has also been tested on clean venvs, is to do:

> pip install hebpipe

If you want to be extra sure there are no dependency conflicts with other libraries you have installed, you should install it in a clean venv, either via pip inside the venv, or using python setup.py install. You shouldn't need to install xrenner or any other depdendency yourself, although of course you can also manually pip install the requirements file.

Does either of those work?

[shaked571]

I am getting the same error, I believe there is an issue in the way you Import from .lib.mtlmodel import Tagger Because then you import:

from lib.crfutils.crf import CRF
from lib.crfutils.viterbi import ViterbiDecoder,ViterbiLoss
from lib.reorder_sgml import reorder
from lib.tt2conll import conllize

But the issue is that there is no such file such as crfutils hence you are getting ModuleNotFoundError calling again to the fallback lib causing the program to crash

Although I do see this directory in the git, I guess some changes in the setup tool didn't include it in the package.

A complete fix if I may suggest would be editing the init.py file as such: lib/init.py

from .tt2conll import  conllize
from .reorder_sgml import reorder
from .dropout import WordDropout,LockedDropout
from .crfutils.crf import CRF
from .crfutils.viterbi import ViterbiDecoder,ViterbiLoss

And to fix the imports in mtlmodel.py (lines 11:18):

from .dropout import WordDropout,LockedDropout
from transformers import BertModel,BertTokenizerFast,BertConfig
from random import sample
from collections import defaultdict
from .crfutils.crf import CRF
from .crfutils.viterbi import ViterbiDecoder,ViterbiLoss
from .reorder_sgml import reorder
from .tt2conll import conllize

[amir-zeldes]

Hm, this is surprising because I tested everything in a clean venv... I suspect these issues pop up when the main script is invoked from a different working directory, so maybe depending on whether it is invoked as a module (python -m hebpipe) or whether the main script is run directly.

In any case I don't see any harm in the imports you're suggesting so I'll just add those, maybe with a try block in case we do get a relative import:

try:
    from .tt2conll import  conllize
    from .reorder_sgml import reorder
    from .dropout import WordDropout,LockedDropout
    from .crfutils.crf import CRF
    from .crfutils.viterbi import ViterbiDecoder,ViterbiLoss
except ModuleNotFoundError:
    from lib.tt2conll import  conllize
    from lib.reorder_sgml import reorder
    from lib.dropout import WordDropout,LockedDropout
    from lib.crfutils.crf import CRF
    from lib.crfutils.viterbi import ViterbiDecoder,ViterbiLoss

That should catch both conditions hopefully. @nitinvwaran just wanted to ping you in case you have any different ideas, see the fix in v3.0.0.2.

[shaked571]

The main thing is that you still miss crfutils when you pip install the package.

I didn't get this folder when I installed it but with a quick debug I could see what was missing, and I came to this Git and download the file manually, but I guess the average Joe wouldn't do the above.

[amir-zeldes]

Thanks for pointing this out! It took me a while to find time to check this, but it looks like crfutils was missing an init file, so python setup sdist was not including it in the PyPI distribution archive. Should be fixed now, but let me know if you notice any issues!