search

lucidrains / audiolm-pytorch

Implementation of AudioLM, a SOTA Language Modeling Approach to Audio Generation out of Google Research, in Pytorch
MIT License
2.36k stars 255 forks source link

EOFError: Ran out of input when trying to train CoarseTransformer #103

Closed adamfils closed 1 year ago

adamfils commented 1 year ago

import torch from audiolm_pytorch import HubertWithKmeans, SoundStream, CoarseTransformer, CoarseTransformerWrapper, \ CoarseTransformerTrainer

wav2vec = HubertWithKmeans( checkpoint_path='./models/hubert_base_ls960.pt', kmeans_path='./models/hubert_base_ls960_L9_km500.bin' )

soundstream = SoundStream( codebook_size=1024, rq_num_quantizers=8, )

soundstream.load('/home/adamfils/Downloads/audiolm/results1/soundstream.11000.pt')

coarse_transformer = CoarseTransformer( num_semantic_tokens=wav2vec.codebook_size, codebook_size=1024, num_coarse_quantizers=3, dim=512, depth=6 )

trainer = CoarseTransformerTrainer( transformer=coarse_transformer, soundstream=soundstream, wav2vec=wav2vec, folder='/home/adamfils/Downloads/LibriSpeech', batch_size=45, data_max_length=320 * 32,

data_max_length_seconds=5,

num_train_steps=10000

)

trainer.train()

Error Stack"

Traceback (most recent call last): File "/home/adamfils/Downloads/audiolm/coarse.py", line 5, in wav2vec = HubertWithKmeans( File "/home/adamfils/Downloads/audiolm/audiolm_pytorch/hubert_kmeans.py", line 44, in init checkpoint = torch.load(checkpoint_path) File "/home/adamfils/.local/lib/python3.10/site-packages/torch/serialization.py", line 795, in load return _legacy_load(opened_file, map_location, pickle_module, pickle_load_args) File "/home/adamfils/.local/lib/python3.10/site-packages/torch/serialization.py", line 1002, in _legacy_load magic_number = pickle_module.load(f, pickle_load_args) EOFError: Ran out of input

lucidrains commented 1 year ago

@adamfils your hubert checkpoint is broken

i would suggest redownloading them from fairseq and making sure they load