Any way to run 5b on a 1080 ti?

[petersandmann]

Is there any way of getting 5b running on a 1080 ti? From my understanding it seems to just have a few megs too few Vram.

I am running python jukebox/sample.py --model=5b --name=sample_5b --levels=3 --sample_length_in_seconds=20 --total_sample_length_in_seconds=180 --sr=44100 --n_samples=3 --hop_fraction=0.5,0.5,0.125

even with reduced n_samples it does not work:

Sampling 6890 tokens for [0,6890]. Conditioning on 0 tokens
Ancestral sampling 3 samples with temp=0.99, top_k=0, top_p=0.0
Traceback (most recent call last):
  File "jukebox/sample.py", line 279, in <module>
    fire.Fire(run)
  File "/home/user/anaconda3/envs/jukebox/lib/python3.7/site-packages/fire/core.py", line 127, in Fire
    component_trace = _Fire(component, args, context, name)
  File "/home/user/anaconda3/envs/jukebox/lib/python3.7/site-packages/fire/core.py", line 366, in _Fire
    component, remaining_args)
  File "/home/user/anaconda3/envs/jukebox/lib/python3.7/site-packages/fire/core.py", line 542, in _CallCallable
    result = fn(*varargs, **kwargs)
  File "jukebox/sample.py", line 276, in run
    save_samples(model, device, hps, sample_hps)
  File "jukebox/sample.py", line 244, in save_samples
    ancestral_sample(labels, sampling_kwargs, priors, hps)
  File "jukebox/sample.py", line 127, in ancestral_sample
    zs = _sample(zs, labels, sampling_kwargs, priors, sample_levels, hps)
  File "jukebox/sample.py", line 102, in _sample
    zs = sample_level(zs, labels[level], sampling_kwargs[level], level, prior, total_length, hop_length, hps)
  File "jukebox/sample.py", line 87, in sample_level
    zs = sample_partial_window(zs, labels, sampling_kwargs, level, prior, total_length, hps)
  File "jukebox/sample.py", line 28, in sample_partial_window
    return sample_single_window(zs, labels, sampling_kwargs, level, prior, start, hps)
  File "jukebox/sample.py", line 69, in sample_single_window
    z_samples_i = prior.sample(n_samples=z_i.shape[0], z=z_i, z_conds=z_conds_i, y=y_i, **sampling_kwargs)
  File "/home/user/jukebox/jukebox/prior/prior.py", line 261, in sample
    x_cond, y_cond, prime = self.get_cond(z_conds, y)
  File "/home/user/jukebox/jukebox/prior/prior.py", line 241, in get_cond
    y_cond, y_pos = self.y_emb(y) if self.y_cond else (None, None)
  File "/home/user/anaconda3/envs/jukebox/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/user/jukebox/jukebox/prior/conditioners.py", line 153, in forward
    pos_emb = self.total_length_emb(total_length) + self.absolute_pos_emb(start, end) + self.relative_pos_emb(start/total_length, end/total_length)
  File "/home/user/anaconda3/envs/jukebox/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/user/jukebox/jukebox/prior/conditioners.py", line 111, in forward
    return self.emb(bins)
  File "/home/user/anaconda3/envs/jukebox/lib/python3.7/site-packages/torch/nn/modules/module.py", line 493, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/user/anaconda3/envs/jukebox/lib/python3.7/site-packages/torch/nn/modules/sparse.py", line 117, in forward
    self.norm_type, self.scale_grad_by_freq, self.sparse)
  File "/home/user/anaconda3/envs/jukebox/lib/python3.7/site-packages/torch/nn/functional.py", line 1506, in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)
RuntimeError: CUDA out of memory. Tried to allocate 450.00 MiB (GPU 0; 10.92 GiB total capacity; 9.60 GiB already allocated; 397.50 MiB free; 160.43 MiB cached)```

[mattwkcc]

Try running it on the lowest prior '1b_lyrics'. Used on a GeForce GTX 1070 NVIDIA-SMI 440.100 CUDA Version: 10.2.

python jukebox/sample.py --model=1b_lyrics --name=1b_sample --levels=3 --sample_length_in_seconds=20 --total_sample_length_in_seconds=180 --sr=44100 --n_samples=3 --hop_fraction=0.5,0.5,0.1250

[peterfranzo]

1b_lyrics works fine.

But won't 1b_lyrics yield inferior results? I was under the impression that 5b was the better model? Also there is no 1b (no lyrics) but I want to generate just random music

thanks for your help!

[tmladek]

Hi, did you ever figure this out? Could using multiple 1080Tis possibly help?