search

jrzaurin / pytorch-widedeep

A flexible package for multimodal-deep-learning to combine tabular data with text and images using Wide and Deep models in Pytorch
Apache License 2.0
1.3k stars 190 forks source link

"TypeError: issubclass() arg 1 must be a class" when spaCy gets imported by fastai_transforms #207

Closed vsahishnu closed 7 months ago

vsahishnu commented 7 months ago

Error faced:

Traceback (most recent call last):
  File "/opt/homebrew/Cellar/python@3.9/3.9.19/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pydoc.py", line 439, in safeimport
    module = __import__(path)
  File "/Users/seth/MyStuff/repo_stuff/PyI/Numeric/src/pred/Seq2SeqFinal/pred_controller.py", line 5, in <module>
    from pred.Seq2SeqFinal.Seq2SeqTransformer import log_test
  File "/Users/seth/MyStuff/repo_stuff/PyI/Numeric/src/pred/Seq2SeqFinal/Seq2SeqTransformer.py", line 15, in <module>
    from Numeric.src.pred.Seq2SeqFinal.sainthandler import SaintHandler
  File "/Users/seth/MyStuff/repo_stuff/PyI/Numeric/src/pred/Seq2SeqFinal/sainthandler.py", line 3, in <module>
    from pytorch_widedeep.preprocessing import TabPreprocessor
  File "/Users/seth/MyStuff/repo_stuff/venvs/numerics/lib/python3.9/site-packages/pytorch_widedeep/__init__.py", line 9, in <module>
    from pytorch_widedeep.utils import (
  File "/Users/seth/MyStuff/repo_stuff/venvs/numerics/lib/python3.9/site-packages/pytorch_widedeep/utils/__init__.py", line 1, in <module>
    from pytorch_widedeep.utils.text_utils import (
  File "/Users/seth/MyStuff/repo_stuff/venvs/numerics/lib/python3.9/site-packages/pytorch_widedeep/utils/text_utils.py", line 7, in <module>
    from pytorch_widedeep.utils.fastai_transforms import (
  File "/Users/seth/MyStuff/repo_stuff/venvs/numerics/lib/python3.9/site-packages/pytorch_widedeep/utils/fastai_transforms.py", line 18, in <module>
    import spacy
  File "/Users/seth/MyStuff/repo_stuff/venvs/numerics/lib/python3.9/site-packages/spacy/__init__.py", line 13, in <module>
    from . import pipeline  # noqa: F401
  File "/Users/seth/MyStuff/repo_stuff/venvs/numerics/lib/python3.9/site-packages/spacy/pipeline/__init__.py", line 1, in <module>
    from .attributeruler import AttributeRuler
  File "/Users/seth/MyStuff/repo_stuff/venvs/numerics/lib/python3.9/site-packages/spacy/pipeline/attributeruler.py", line 8, in <module>
    from ..language import Language
  File "/Users/seth/MyStuff/repo_stuff/venvs/numerics/lib/python3.9/site-packages/spacy/language.py", line 43, in <module>
    from .pipe_analysis import analyze_pipes, print_pipe_analysis, validate_attrs
  File "/Users/seth/MyStuff/repo_stuff/venvs/numerics/lib/python3.9/site-packages/spacy/pipe_analysis.py", line 6, in <module>
    from .tokens import Doc, Span, Token
  File "/Users/seth/MyStuff/repo_stuff/venvs/numerics/lib/python3.9/site-packages/spacy/tokens/__init__.py", line 1, in <module>
    from ._serialize import DocBin
  File "/Users/seth/MyStuff/repo_stuff/venvs/numerics/lib/python3.9/site-packages/spacy/tokens/_serialize.py", line 14, in <module>
    from ..vocab import Vocab
  File "spacy/vocab.pyx", line 1, in init spacy.vocab
  File "spacy/tokens/doc.pyx", line 49, in init spacy.tokens.doc
  File "/Users/seth/MyStuff/repo_stuff/venvs/numerics/lib/python3.9/site-packages/spacy/schemas.py", line 287, in <module>
    class TokenPattern(BaseModel):
  File "pydantic/main.py", line 198, in pydantic.main.ModelMetaclass.__new__
  File "pydantic/fields.py", line 506, in pydantic.fields.ModelField.infer
  File "pydantic/fields.py", line 436, in pydantic.fields.ModelField.__init__
  File "pydantic/fields.py", line 552, in pydantic.fields.ModelField.prepare
  File "pydantic/fields.py", line 661, in pydantic.fields.ModelField._type_analysis
  File "pydantic/fields.py", line 668, in pydantic.fields.ModelField._type_analysis
  File "/opt/homebrew/Cellar/python@3.9/3.9.19/Frameworks/Python.framework/Versions/3.9/lib/python3.9/typing.py", line 852, in __subclasscheck__
    return issubclass(cls, self.__origin__)
TypeError: issubclass() arg 1 must be a class

I am using: python==3.9 pytorch-widedeep==1.4.0 & 1.5.1 (tried both) env == regular venv and conda (tried both)

I installed pytorch-widedeep in an env along with a list of other packages which DID have version clashes. I played around with the versions and got the packages installed. But I kept receiving the same above error whenever I tried to use TabPreprocessor.

I played around with the env types and packages versions (though I installed pytorch-widedeep along with my other packages...) @jrzaurin suggested to check out this stackoverflow article: https://stackoverflow.com/questions/77037891/typeerror-issubclass-arg-1-must-be-a-class which led to a solved issue: https://github.com/explosion/spaCy/issues/12659 which is related to versions of pydantic and spaCy. the solution was to keep the version of typing_extensions<4.6.0

(potential solution:) So I ended up using pandas==2.0.0 torchvision==0.15.0 typing_extensions==4.5.0 then installed pytorch-widedeep==1.5.1

The above config got rid of the issue with the typing_extensions version change...

Later @5uperpalo pointed out a fresh install with latest python and pytorch-widedeep works without issues. I tried it out with a fresh python install using both regular venv and conda and installed only pytorch-widedeep, both of which worked without issues.

Probable reason: spaCy's dependencies version getting mixed up due to other packages I was installing

Huge thanks to @jrzaurin and @5uperpalo for looking into the issue and helping me solve it :)

jrzaurin commented 7 months ago

Thanks @vsahishnu

closing the issue. Thanks for the write up