Open InderjeetVishnoi opened 2 months ago
Ok that's a weird issue - do you know if normal HF loading works?
Yes, HF loading works fine. The key observation is : even loading from local works but its not working when a tar file is created from the model artifacts and is extracted back before loading.
Ok will try investigating this - sorry on the issue!
Hi Do we have any progress or update on this? Or if you could provide some workaround for meanwhile?
Sorry been a bit busy apologies! Weird it should normally work since I detect a local inventory - how about passing just "model"
and not "./model"
I have tried that too , does not work. In-fact , recently encountered the same issue even without creating tar and extracting it. Steps are as below :
Hi @danielhanchen Do we have any workaround for this, I am kind of blocked because of this issue. The moment model artifacts are placed form point A to point B, loading it from B doesn't works 🥲
I created a tar file out of a unsloth fine-tuned model(base-model: unsloth/gemma-2b-bnb-4bit) using PEFT and pushed it to gcsBucket. I am downloading the artifacts from gcs bucket, extracting the files and locally saving them into a folder. Now, I am trying to load the model using this local directory path, but getting error : Exception: expected value at line 1 column 1
How can I load the model from the given directory?