kongds / scaling_sentemb

Scaling Sentence Embeddings with Large Language Models
85 stars 4 forks source link

opt-1.3b does not have a file config.json #7

Closed JhonDan1999 closed 9 months ago

JhonDan1999 commented 9 months ago

I am trying to use the model but it gave me this error Screenshot 2023-10-01 at 7 00 26 PM Screenshot 2023-10-01 at 7 01 02 PM

Screenshot 2023-10-01 at 7 00 45 PM

can you please help me with the correct way to use the model with huggingface

kongds commented 9 months ago

The models we uploaded are Parameter-Efficient Fine-Tuning (PEFT) checkpoints.

For example To use prompted-opt-1.3b, you need load opt-1.3b first, and then load peft checkpoint over opt-1.3b

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer
from peft import PeftModel

# Import our models. The package will take care of downloading the models automatically
tokenizer = AutoTokenizer.from_pretrained("facebook/opt-1.3b")
model = AutoModelForCausalLM.from_pretrained("facebook/opt-1.3b")
tokenizer.pad_token_id = 0 
tokenizer.padding_side = "left"
model = PeftModel.from_pretrained(model, "royokong/prompteol-opt-1.3b", torch_dtype=torch.float16)