unslothai / unsloth

Finetune Llama 3.2, Mistral, Phi & Gemma LLMs 2-5x faster with 80% less memory
https://unsloth.ai
Apache License 2.0
17.49k stars 1.21k forks source link

NameError: name 'os' is not defined #927

Open Tizzzzy opened 2 months ago

Tizzzzy commented 2 months ago

Hi, I am trying to run the Llama-3.1 8b + Unsloth 2x faster finetuning.ipynb you provided in the README. However, when I use google colab to run the second cell I got this error:

---------------------------------------------------------------------------
NameError                                 Traceback (most recent call last)
[<ipython-input-5-cc43f0184832>](https://localhost:8080/#) in <cell line: 1>()
----> 1 from unsloth import FastLanguageModel
      2 import torch
      3 max_seq_length = 2048 # Choose any! We auto support RoPE Scaling internally!
      4 dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
      5 load_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False.

[/usr/local/lib/python3.10/dist-packages/unsloth/__init__.py](https://localhost:8080/#) in <module>
     30 # We do have a beta version, which you can contact us about!
     31 # Thank you for your understanding and we appreciate it immensely!
---> 32 if "CUDA_VISIBLE_DEVICES" in os.environ:
     33     os.environ["CUDA_DEVICE_ORDER"] = "PCI_BUS_ID"
     34     devices = os.environ["CUDA_VISIBLE_DEVICES"]

NameError: name 'os' is not defined

The second cell code is:

from unsloth import FastLanguageModel
import torch
max_seq_length = 2048 # Choose any! We auto support RoPE Scaling internally!
dtype = None # None for auto detection. Float16 for Tesla T4, V100, Bfloat16 for Ampere+
load_in_4bit = True # Use 4bit quantization to reduce memory usage. Can be False.

# 4bit pre quantized models we support for 4x faster downloading + no OOMs.
fourbit_models = [
    "unsloth/Meta-Llama-3.1-8B-bnb-4bit",      # Llama-3.1 15 trillion tokens model 2x faster!
    "unsloth/Meta-Llama-3.1-8B-Instruct-bnb-4bit",
    "unsloth/Meta-Llama-3.1-70B-bnb-4bit",
    "unsloth/Meta-Llama-3.1-405B-bnb-4bit",    # We also uploaded 4bit for 405b!
    "unsloth/Mistral-Nemo-Base-2407-bnb-4bit", # New Mistral 12b 2x faster!
    "unsloth/Mistral-Nemo-Instruct-2407-bnb-4bit",
    "unsloth/mistral-7b-v0.3-bnb-4bit",        # Mistral v3 2x faster!
    "unsloth/mistral-7b-instruct-v0.3-bnb-4bit",
    "unsloth/Phi-3-mini-4k-instruct",          # Phi-3 2x faster!d
    "unsloth/Phi-3-medium-4k-instruct",
    "unsloth/gemma-2-9b-bnb-4bit",
    "unsloth/gemma-2-27b-bnb-4bit",            # Gemma 2x faster!
] # More models at https://huggingface.co/unsloth

model, tokenizer = FastLanguageModel.from_pretrained(
    model_name = "unsloth/Meta-Llama-3.1-8B",
    max_seq_length = max_seq_length,
    dtype = dtype,
    load_in_4bit = load_in_4bit,
    # token = "hf_...", # use one if using gated models like meta-llama/Llama-2-7b-hf
)
danielhanchen commented 2 months ago

Whoopsies - just fixed!!! (Press Runtime then Disconnect and Delete Runtime and press Run again!)