"OSError: could not get source code" on the phi3 colab script

[txhno]

==((====))==  Unsloth: Fast Mistral patching release 2024.7
   \\   /|    GPU: Tesla T4. Max memory: 14.748 GB. Platform = Linux.
O^O/ \_/ \    Pytorch: 2.3.0+cu121. CUDA = 7.5. CUDA Toolkit = 12.1.
\        /    Bfloat16 = FALSE. FA [Xformers = 0.0.26.post1. FA2 = False]
 "-____-"     Free Apache license: http://github.com/unslothai/unsloth
---------------------------------------------------------------------------
OSError                                   Traceback (most recent call last)
[<ipython-input-7-7adf35ab4d26>](https://localhost:8080/#) in <cell line: 8>()
      6 load_in_4bit = True
      7 
----> 8 model, tokenizer = FastLanguageModel.from_pretrained(
      9     model_name="lora_model",
     10     max_seq_length=max_seq_length,

7 frames
[/usr/local/lib/python3.10/dist-packages/unsloth/models/loader.py](https://localhost:8080/#) in from_pretrained(model_name, max_seq_length, dtype, load_in_4bit, token, device_map, rope_scaling, fix_tokenizer, trust_remote_code, use_gradient_checkpointing, resize_model_vocab, revision, *args, **kwargs)
    170         pass
    171 
--> 172         model, tokenizer = dispatch_model.from_pretrained(
    173             model_name        = model_name,
    174             max_seq_length    = max_seq_length,

[/usr/local/lib/python3.10/dist-packages/unsloth/models/mistral.py](https://localhost:8080/#) in from_pretrained(model_name, max_seq_length, dtype, load_in_4bit, token, device_map, rope_scaling, fix_tokenizer, model_patcher, tokenizer_name, trust_remote_code, **kwargs)
    315         **kwargs,
    316     ):
--> 317         return FastLlamaModel.from_pretrained(
    318             model_name        = model_name,
    319             max_seq_length    = max_seq_length,

[/usr/local/lib/python3.10/dist-packages/unsloth/models/llama.py](https://localhost:8080/#) in from_pretrained(model_name, max_seq_length, dtype, load_in_4bit, token, device_map, rope_scaling, fix_tokenizer, model_patcher, tokenizer_name, trust_remote_code, **kwargs)
   1129            f' "-____-"     Free Apache license: http://github.com/unslothai/unsloth'
   1130         print(statistics)
-> 1131         model_patcher.pre_patch()
   1132         get_statistics() # For debugging - we use a download counter to see if environments are not breaking
   1133 

[/usr/local/lib/python3.10/dist-packages/unsloth/models/mistral.py](https://localhost:8080/#) in pre_patch()
    272     @staticmethod
    273     def pre_patch():
--> 274         init_name, function = patch_linear_scaling(
    275             model_name         = "mistral",
    276             rope_module        = LlamaRotaryEmbedding,

[/usr/local/lib/python3.10/dist-packages/unsloth/models/_utils.py](https://localhost:8080/#) in patch_linear_scaling(model_name, rope_module, scaled_rope_module, attention_module)
    619         f"{model_name.title()}Attention, {model_name.title()}Config"
    620 
--> 621     function = inspect.getsource(attention_module.__init__)
    622     where = function.find("def")
    623     function = function.split("\n")

[/usr/lib/python3.10/inspect.py](https://localhost:8080/#) in getsource(object)
   1137     or code object.  The source code is returned as a single string.  An
   1138     OSError is raised if the source code cannot be retrieved."""
-> 1139     lines, lnum = getsourcelines(object)
   1140     return ''.join(lines)
   1141 

[/usr/lib/python3.10/inspect.py](https://localhost:8080/#) in getsourcelines(object)
   1119     raised if the source code cannot be retrieved."""
   1120     object = unwrap(object)
-> 1121     lines, lnum = findsource(object)
   1122 
   1123     if istraceback(object):

[/usr/lib/python3.10/inspect.py](https://localhost:8080/#) in findsource(object)
    956         lines = linecache.getlines(file)
    957     if not lines:
--> 958         raise OSError('could not get source code')
    959 
    960     if ismodule(object):

OSError: could not get source code

[danielhanchen]

Apologies - hopefully I fixed it in Colab! Please Disconnect and Restart Runtime in Colab to try it again - thanks for the report!

[txhno]

Apologies - hopefully I fixed it in Colab! Please Disconnect and Restart Runtime in Colab to try it again - thanks for the report!

thanks! will try it out

[txhno]

Works now! Also i wanted to know if we can run inference on CPU?