search

yangdongchao / LLM-Codec

The open source code for LLM-Codec
99 stars 2 forks source link

missing file local_embedding_path: embed_llama2.pt #2

Open nkundiushuti opened 3 weeks ago

nkundiushuti commented 3 weeks ago

there is an error when trying to load the model the error is in the model itself checkpoint = torch.load(local_embedding_path, map_location="cpu")['weight'] this apparently expects embed_llama2.pt not included in the repository, also not in the llama2 model files.

yangdongchao commented 3 weeks ago

there is an error when trying to load the model the error is in the model itself checkpoint = torch.load(local_embedding_path, map_location="cpu")['weight'] this apparently expects embed_llama2.pt not included in the repository, also not in the llama2 model files.

Please refer to wget https://huggingface.co/Dongchao/UniAudio/resolve/main/embed_llama2.pt

nkundiushuti commented 3 weeks ago

thanks for uploading the file and for the quick reply! I believe there is an issue in the checkpoint you provided. according to the paper and the github code, the first codebook has the size 3248 (not being initialized with the llama2 codebook). however, the size of the 'quantizer.quantizers.0.codebook.weight' is 32000, same as the llama2 codebook. (parameter_dict['codec_model']['quantizer.quantizers.0.codebook.weight'].shape ). this results in an error. maybe this is not the correct checkpoint? https://huggingface.co/Dongchao/UniAudio/blob/main/llm3_codec_uni.pth

yangdongchao commented 2 weeks ago

thanks for uploading the file and for the quick reply! I believe there is an issue in the checkpoint you provided. according to the paper and the github code, the first codebook has the size 3248 (not being initialized with the llama2 codebook). however, the size of the 'quantizer.quantizers.0.codebook.weight' is 32000, same as the llama2 codebook. (parameter_dict['codec_model']['quantizer.quantizers.0.codebook.weight'].shape ). this results in an error. maybe this is not the correct checkpoint? https://huggingface.co/Dongchao/UniAudio/blob/main/llm3_codec_uni.pth

Hi, sorry for the delay. let me check it again tomorrow.

nkundiushuti commented 2 weeks ago

thanks! it works with this checkpoint but with using the same codebook as llama for all the 3 levels. it would be good to have the model with the semantic RVQ.

yangdongchao commented 1 week ago

thanks! it works with this checkpoint but with using the same codebook as llama for all the 3 levels. it would be good to have the model with the semantic RVQ.

Please refer to wget https://huggingface.co/Dongchao/UniAudio/resolve/main/3layer.pth

nkundiushuti commented 1 week ago

I think this one is the same as the above, level 0 has 32000 tokens not 3248