danielhanchen
commented
7 hours ago Wait why is there a ".json" randomly in the search for the directories - do you have a folder called "unsloth/Phi-3.5-mini-instruct-bnb-4bit"?
Open ygl1020 opened 1 day ago
danielhanchen
commented
7 hours ago Wait why is there a ".json" randomly in the search for the directories - do you have a folder called "unsloth/Phi-3.5-mini-instruct-bnb-4bit"?
I successfully installed Unsloth locally on a Windows system and attempted to test the following example code snippet:
Code Snippet
from unsloth import FastLanguageModel
max_seq_length = 2048 # Choose any! RoPE Scaling is auto-supported internally. dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+. load_in_4bit = True # Use 4-bit quantization to reduce memory usage. Can be False.
model, tokenizer = FastLanguageModel.from_pretrained( model_name="unsloth/Phi-3.5-mini-instruct-bnb-4bit", max_seq_length=max_seq_length, dtype=dtype, load_in_4bit=load_in_4bit,
token="hf_...", # Optional for gated models like meta-llama/Llama-2-7b-hf
)
However, running the script produced an error.
Error Log plaintext Copy code (.venv) PS C:\ScoreProjectTesting\unsloth\triton> python C:\ScoreProjectTesting\unsloth\test.py 🦥 Unsloth: Will patch your computer to enable 2x faster free fine-tuning. unsloth/Phi-3.5-mini-instruct-bnb-4bit*.json Traceback (most recent call last): File "C:\ScoreProjectTesting\unsloth.venv\lib\site-packages\huggingface_hub\hf_file_system.py", line 121, in _repo_and_revision_exist self._api.repo_info( File "C:\ScoreProjectTesting\unsloth.venv\lib\site-packages\huggingface_hub\utils_validators.py", line 106, in _inner_fn validate_repo_id(arg_value) File "C:\ScoreProjectTesting\unsloth.venv\lib\site-packages\huggingface_hub\utils_validators.py", line 160, in validate_repo_id raise HFValidationError( huggingfacehub.errors.HFValidationError: Repo id must use alphanumeric chars or '-', '', '.', '--' and '..' are forbidden, '-' and '.' cannot start or end the name, max length is 96: 'unsloth/Phi-3.5-mini-instruct-bnb-4bit*.json'.
The above exception was the direct cause of the following exception:
Traceback (most recent call last): File "C:\ScoreProjectTesting\unsloth\test.py", line 9, in
model, tokenizer = FastLanguageModel.from_pretrained(
File "C:\ScoreProjectTesting\unsloth.venv\lib\site-packages\unsloth\models\loader.py", line 231, in from_pretrained
files = HfFileSystem(token=token).glob(os.path.join(model_name, ".json"))
File "C:\ScoreProjectTesting\unsloth.venv\lib\site-packages\huggingface_hub\hf_file_system.py", line 408, in glob
path = self.resolve_path(path, revision=kwargs.get("revision")).unresolve()
File "C:\ScoreProjectTesting\unsloth.venv\lib\site-packages\huggingface_hub\hf_file_system.py", line 193, in resolve_path
_raise_file_not_found(path, err)
File "C:\ScoreProjectTesting\unsloth.venv\lib\site-packages\huggingface_hub\hf_file_system.py", line 881, in _raise_file_not_found
raise FileNotFoundError(msg) from err
FileNotFoundError: unsloth/Phi-3.5-mini-instruct-bnb-4bit\.json (invalid repository id)
python version: 3.10.9 operation system: Windows 11 installed library dependence:
Package Version
accelerate 1.1.1 aiohappyeyeballs 2.4.3 aiohttp 3.11.6 aiosignal 1.3.1 async-timeout 5.0.1 attrs 24.2.0 bitsandbytes 0.44.1 certifi 2024.8.30 charset-normalizer 3.4.0 cmake 3.31.0.1 colorama 0.4.6 datasets 3.1.0 dill 0.3.8 docstring_parser 0.16 filelock 3.16.1 frozenlist 1.5.0 fsspec 2024.9.0 hf_transfer 0.1.8 huggingface-hub 0.26.2 idna 3.10 Jinja2 3.1.4 markdown-it-py 3.0.0 MarkupSafe 3.0.2 mdurl 0.1.2 mpmath 1.3.0 multidict 6.1.0 multiprocess 0.70.16 networkx 3.4.2 ninja 1.11.1.1 numpy 2.1.3 packaging 24.2 pandas 2.2.3 peft 0.13.2 pillow 10.2.0 pip 24.3.1 propcache 0.2.0 protobuf 3.20.3 psutil 6.1.0 pyarrow 18.0.0 pybind11 2.13.6 Pygments 2.18.0 python-dateutil 2.9.0.post0 pytz 2024.2 PyYAML 6.0.2 regex 2024.11.6 requests 2.32.3 rich 13.9.4 safetensors 0.4.5 sentencepiece 0.2.0 setuptools 65.5.0 shtab 1.7.1 six 1.16.0 sympy 1.13.1 tokenizers 0.20.3 torch 2.5.1+cu118 torchaudio 2.5.1+cu118 torchvision 0.20.1+cu118 tqdm 4.67.0 transformers 4.46.3 triton 2.1.0 trl 0.12.1 typing_extensions 4.12.2 tyro 0.9.1 tzdata 2024.2 unsloth 2024.11.7 unsloth_zoo 2024.11.5 urllib3 2.2.3 wheel 0.45.0 xformers 0.0.28.post3 xxhash 3.5.0 yarl 1.17.2
Tried solution: updated unsloth library by command: pip install --upgrade --no-cache-dir "unsloth[colab-new] @ git+https://github.com/unslothai/unsloth.git"