I downloaded the llamada2 to my local disk, but i still cannot run your code, The error message is: "TypeError: Object of type LlamaTokenizerFast is not JSON serializable."

[zob109]

There is my issue: /home/fountain/miniconda3/bin/conda run -n Linear_Alignment --no-capture-output python /home/fountain/pycharmProjects/Linear_Alignment/demo.py Loading checkpoint shards: 100%|██████████████████| 2/2 [00:17<00:00, 8.77s/it] Traceback (most recent call last): File "/home/fountain/pycharmProjects/Linear_Alignment/demo.py", line 134, in model = ConstractiveDecodingModel.from_pretrained( File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/site-packages/transformers/modeling_utils.py", line 3510, in from_pretrained model.generation_config = GenerationConfig.from_pretrained( File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/site-packages/transformers/generation/configuration_utils.py", line 753, in from_pretrained config = cls.from_dict(config_dict, **kwargs) File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/site-packages/transformers/generation/configuration_utils.py", line 791, in from_dict logger.info(f"Generate config {config}") File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/site-packages/transformers/generation/configuration_utils.py", line 350, in repr return f"{self.class.name} {self.to_json_string(ignore_metadata=True)}" File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/site-packages/transformers/generation/configuration_utils.py", line 876, in to_json_string return json.dumps(config_dict, indent=2, sort_keys=True) + "\n" File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/json/init.py", line 234, in dumps return cls( File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/json/encoder.py", line 201, in encode chunks = list(chunks) File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/json/encoder.py", line 431, in _iterencode yield from _iterencode_dict(o, _current_indent_level) File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/json/encoder.py", line 405, in _iterencode_dict yield from chunks File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/json/encoder.py", line 438, in _iterencode o = _default(o) File "/home/fountain/miniconda3/envs/Linear_Alignment/lib/python3.9/json/encoder.py", line 179, in default raise TypeError(f'Object of type {o.class.name} ' TypeError: Object of type LlamaTokenizerFast is not JSON serializable ERROR conda.cli.main_run:execute(125): conda run python /home/fountain/pycharmProjects/Linear_Alignment/demo.py failed. (See above for error) How to solve this question?

[zob109]

my enviroment are follows: Ubuntu 22.04.3, Python 3.9.19

[sod1010]

Have you solved this problem already? I encountered the same problem.

[Linzy-Liu]

I've solved the problem. The problem is caused by the extra parameter tokenzier in model initialization. The tokenizer will be passed to the kwargs param in from_pretained method and finally reach the json.dump method, where the error occurs.

The solution to the problem is to remove the tokenizer param in initialization ,and add a custom from_pretrained method to seperate the intialization of Llama and tokenizer. Here is my solution:

class ConstractiveDecodingModel(LlamaForCausalLM):
    _tied_weights_keys = ["lm_head.weight"]

    def __init__(self, config):
        super().__init__(config)
        self.tokenizer = None  # Initialize tokenizer separately

    @classmethod
    def from_pretrained(cls, pretrained_model_name_or_path, tokenizer=None, **kwargs):
        # Load the model without leaving the tokenizer in kwargs
        model = super().from_pretrained(pretrained_model_name_or_path, **kwargs)

        if tokenizer is not None:  # Attach tokenizer separately
            model.tokenizer = tokenizer
        return model

[sod1010]

Thank you so much~ I have solved this BUG through your method!