Closed lsrodri closed 1 year ago
Thanks @lucasiq for providing the stack trace, the problem was a type annotation not compatible with python 3.7. galai==1.1.2 should fix the issue.
Thanks @lucasiq for providing the stack trace, the problem was a type annotation not compatible with python 3.7. galai==1.1.2 should fix the issue.
Thank you for your answer. I have updated it, galai can now be imported, but the rest stops at the model assignment. I'm running this example on a 3090:
import galai as gal
model = gal.load_model("standard")
and I'm getting this error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Users\admin\AppData\Local\Programs\Python\Python37\lib\site-packages\galai\__init__.py", line 124, in load_model
model._load_checkpoint(checkpoint_path=hf_model)
File "C:\Users\admin\AppData\Local\Programs\Python\Python37\lib\site-packages\galai\model.py", line 114, in _load_checkpoint
max_memory=max_memory,
File "C:\Users\admin\AppData\Local\Programs\Python\Python37\lib\site-packages\transformers\modeling_utils.py", line 2406, in from_pretrained
dispatch_model(model, device_map=device_map, offload_dir=offload_folder, offload_index=offload_index)
TypeError: dispatch_model() got an unexpected keyword argument 'offload_index'
I tried to run base and mini as well, but the error is the same. Is it advisable to run this on Python 3.9? I thought this was restricted to 3.7.8
Python 3.7 should be ok since galai 1.1.2. The issue looks like incompatibility between transformers
and accelerate
. offload_index
was added in transformers==v4.25.1 and accelerate==v0.15.0, so the easiest should be to upgrade the later.
Thanks! Updating accelerate allowed me to get the model to load.
I recently updated to galai 1.1.0, and now I can't go past import:
>>> import galai as gal
Traceback (most recent call last): File "<stdin>", line 1, in <module> File "C:\Users\Lucas\AppData\Local\Programs\Python\Python37\lib\site-packages\galai\__init__.py", line 3, in <module> from galai.model import Model File "C:\Users\Lucas\AppData\Local\Programs\Python\Python37\lib\site-packages\galai\model.py", line 11, in <module> from galai.utils import escape_custom_split_sequence File "C:\Users\Lucas\AppData\Local\Programs\Python\Python37\lib\site-packages\galai\utils.py", line 66, in <module> @dataclass File "C:\Users\Lucas\AppData\Local\Programs\Python\Python37\lib\site-packages\galai\utils.py", line 116, in ModelInfo def all() -> list["ModelInfo"]: TypeError: 'type' object is not subscriptable
After that, I have tried reinstalling transformers, accelerate, tokenizers, and huggingface-hub, but nothing worked so far.
Thank you for your time