Galai import failing after 1.1.0 update

[lsrodri]

I recently updated to galai 1.1.0, and now I can't go past import:

>>> import galai as gal Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Users\Lucas\AppData\Local\Programs\Python\Python37\lib\site-packages\galai\__init__.py", line 3, in <module> from galai.model import Model File "C:\Users\Lucas\AppData\Local\Programs\Python\Python37\lib\site-packages\galai\model.py", line 11, in <module> from galai.utils import escape_custom_split_sequence File "C:\Users\Lucas\AppData\Local\Programs\Python\Python37\lib\site-packages\galai\utils.py", line 66, in <module> @dataclass File "C:\Users\Lucas\AppData\Local\Programs\Python\Python37\lib\site-packages\galai\utils.py", line 116, in ModelInfo def all() -> list["ModelInfo"]: TypeError: 'type' object is not subscriptable

After that, I have tried reinstalling transformers, accelerate, tokenizers, and huggingface-hub, but nothing worked so far.

Thank you for your time

[mkardas]

Thanks @lucasiq for providing the stack trace, the problem was a type annotation not compatible with python 3.7. galai==1.1.2 should fix the issue.

[lsrodri]

Thanks @lucasiq for providing the stack trace, the problem was a type annotation not compatible with python 3.7. galai==1.1.2 should fix the issue.

Thank you for your answer. I have updated it, galai can now be imported, but the rest stops at the model assignment. I'm running this example on a 3090:

import galai as gal
model = gal.load_model("standard")

and I'm getting this error:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "C:\Users\admin\AppData\Local\Programs\Python\Python37\lib\site-packages\galai\__init__.py", line 124, in load_model
    model._load_checkpoint(checkpoint_path=hf_model)
  File "C:\Users\admin\AppData\Local\Programs\Python\Python37\lib\site-packages\galai\model.py", line 114, in _load_checkpoint
    max_memory=max_memory,
  File "C:\Users\admin\AppData\Local\Programs\Python\Python37\lib\site-packages\transformers\modeling_utils.py", line 2406, in from_pretrained
    dispatch_model(model, device_map=device_map, offload_dir=offload_folder, offload_index=offload_index)
TypeError: dispatch_model() got an unexpected keyword argument 'offload_index'

I tried to run base and mini as well, but the error is the same. Is it advisable to run this on Python 3.9? I thought this was restricted to 3.7.8

[mkardas]

Python 3.7 should be ok since galai 1.1.2. The issue looks like incompatibility between transformers and accelerate. offload_index was added in transformers==v4.25.1 and accelerate==v0.15.0, so the easiest should be to upgrade the later.

[lsrodri]

Thanks! Updating accelerate allowed me to get the model to load.