search

nixified-ai / flake

A Nix flake for many AI projects
GNU Affero General Public License v3.0
624 stars 69 forks source link

How to make `text-gen` use a newer git version of ` huggingface/transformers`? #87

Open futu2 opened 4 months ago

futu2 commented 4 months ago

Since some newer models like starcoder2 are not included in model config and when load model it would complain:

  File "/nix/store/7hpffz24mjm12y5ymd2is43lxl7nf27b-python3-3.11.6-env/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 1064, in from_pretrained
    config_class = CONFIG_MAPPING[config_dict["model_type"]]
                   ~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nix/store/7hpffz24mjm12y5ymd2is43lxl7nf27b-python3-3.11.6-env/lib/python3.11/site-packages/transformers/models/auto/configuration_auto.py", line 761, in __getitem__
    raise KeyError(key)
KeyError: 'starcoder2'

In normal env, pip install --upgrade git+https://github.com/huggingface/transformers would work.

max-privatevoid commented 4 months ago

Right now we're using transformers from nixpkgs, which isn't a new enough version to support starcoder2 yet. We could package our own version of transformers.