nlp-uoregon / trankit

Trankit is a Light-Weight Transformer-based Python Toolkit for Multilingual Natural Language Processing
Apache License 2.0
724 stars 99 forks source link

Trankit does not work on new versions of Python #73

Closed fergusq closed 10 months ago

fergusq commented 1 year ago

Trying to use Trankit on Python 3.11 yields this error:

>>> from trankit import Pipeline
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/opt/homebrew/lib/python3.11/site-packages/trankit/__init__.py", line 1, in <module>
    from .pipeline import Pipeline
  File "/opt/homebrew/lib/python3.11/site-packages/trankit/pipeline.py", line 2, in <module>
    from .models.base_models import Multilingual_Embedding
  File "/opt/homebrew/lib/python3.11/site-packages/trankit/models/__init__.py", line 1, in <module>
    from .classifiers import *
  File "/opt/homebrew/lib/python3.11/site-packages/trankit/models/classifiers.py", line 2, in <module>
    from .base_models import *
  File "/opt/homebrew/lib/python3.11/site-packages/trankit/models/base_models.py", line 1, in <module>
    from ..adapter_transformers import AdapterType, XLMRobertaModel
  File "/opt/homebrew/lib/python3.11/site-packages/trankit/adapter_transformers/__init__.py", line 31, in <module>
    from .configuration_auto import ALL_PRETRAINED_CONFIG_ARCHIVE_MAP, CONFIG_MAPPING, AutoConfig
  File "/opt/homebrew/lib/python3.11/site-packages/trankit/adapter_transformers/configuration_auto.py", line 23, in <module>
    from .configuration_bert import BERT_PRETRAINED_CONFIG_ARCHIVE_MAP, BertConfig
  File "/opt/homebrew/lib/python3.11/site-packages/trankit/adapter_transformers/configuration_bert.py", line 21, in <module>
    from .adapter_config import ModelAdaptersConfig
  File "/opt/homebrew/lib/python3.11/site-packages/trankit/adapter_transformers/adapter_config.py", line 121, in <module>
    @dataclass
     ^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/dataclasses.py", line 1230, in dataclass
    return wrap(cls)
           ^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/dataclasses.py", line 1220, in wrap
    return _process_class(cls, init, repr, eq, order, unsafe_hash,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/dataclasses.py", line 958, in _process_class
    cls_fields.append(_get_field(cls, name, type, kw_only))
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/python@3.11/3.11.4_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/dataclasses.py", line 815, in _get_field
    raise ValueError(f'mutable default {type(f.default)} for field '
ValueError: mutable default <class 'trankit.adapter_transformers.adapter_config.InvertibleAdapterConfig'> for field invertible_adapter is not allowed: use default_factory

This is because newer versions of Python determine object mutability differently than previous versions: it checks whether the object is hashable or not, while the previous versions merely checked if the object is a subclass of dict, list or set. See https://bugs.python.org/issue44674

As InvertibleAdapterConfig is immutable, it a __hash__ method should be implemented for it.

alxshine commented 1 year ago

I have the same issue on Python 3.11.3 The PR with a simple fix is linked.