How to load the tokenizer from checkpoint ?

[iwinterknight]

I want to export the trained model as pytorch checkpoint. For that do i need to load the model before running export_state_dict_checkpoint.py ?

If so, when I try to load the model like : checkpoint = '/content/gdrive/MyDrive/Projects/alpaca_lora/alpaca-lora/lora-alpaca/checkpoint-800' tokenizer = LlamaTokenizer.from_pretrained(checkpoint) model, tokenizer, prompter = load(lora_weights=checkpoint, tokenizer=tokenizer)

where load() is :

def load(
    load_8bit: bool = False,
    base_model: str = "decapoda-research/llama-7b-hf",
    lora_weights: str = "tloen/alpaca-lora-7b",
    prompt_template: str = "",  # The prompt template to use, will default to alpaca.
    server_name: str = "0.0.0.0",  # Allows to listen on all interfaces by providing '0.
    share_gradio: bool = False,
    tokenizer: str = None
):
    base_model = base_model or os.environ.get("BASE_MODEL", "")
    assert (
        base_model
    ), "Please specify a --base_model, e.g. --base_model='decapoda-research/llama-7b-hf'"

    prompter = Prompter(prompt_template)
    if not tokenizer:
      tokenizer = LlamaTokenizer.from_pretrained(base_model)
    if device == "cuda":
        model = LlamaForCausalLM.from_pretrained(
            base_model,
            load_in_8bit=load_8bit,
            torch_dtype=torch.float16,
            device_map="auto",
        )
        model = PeftModel.from_pretrained(
            model,
            lora_weights,
            torch_dtype=torch.float16,
        )
    elif device == "mps":
        model = LlamaForCausalLM.from_pretrained(
            base_model,
            device_map={"": device},
            torch_dtype=torch.float16,
        )
        model = PeftModel.from_pretrained(
            model,
            lora_weights,
            device_map={"": device},
            torch_dtype=torch.float16,
        )
    else:
        model = LlamaForCausalLM.from_pretrained(
            base_model, device_map={"": device}, low_cpu_mem_usage=True
        )
        model = PeftModel.from_pretrained(
            model,
            lora_weights,
            device_map={"": device},
        )

    # unwind broken decapoda-research config
    model.config.pad_token_id = tokenizer.pad_token_id = 0  # unk
    model.config.bos_token_id = 1
    model.config.eos_token_id = 2

    if not load_8bit:
        model.half()  # seems to fix bugs for some users.

    model.eval()
    if torch.__version__ >= "2" and sys.platform != "win32":
        model = torch.compile(model)

    return model, tokenizer, prompter

I get an error :

OSError: Can't load tokenizer for '/content/gdrive/MyDrive/Projects/alpaca_lora/alpaca-lora/lora-alpaca/checkpoint-800'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure '/content/gdrive/MyDrive/Projects/alpaca_lora/alpaca-lora/lora-alpaca/checkpoint-800' is the correct path to a directory containing all relevant files for a LlamaTokenizer tokenizer.

[AngainorDev]

Load the tokenizer from the base model (llama) not from your checkpoint.

[larawehbe]

Load the tokenizer from the base model (llama) not from your checkpoint.

So i load tokenizer from base model and base_model from checkpoint ?

[TrieuLe0801]

Load the tokenizer from the base model (llama) not from your checkpoint.

So i load tokenizer from base model and base_model from checkpoint ?

I saw the output folder did not contain the tokenizer file, so it is only used for load checkpoint, not include the tokenizer, and I have to load the pre-trained again to have tokenizer. My solution is saving tokenizer after training with tokenizer.save_pretrained("MY_OUTPUT_DIRECTORY").

I hope it will help you