[ ] An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
[X] My own task or dataset (give details below)
Reproduction
Code:
from bark import SAMPLE_RATE, generate_audio, preload_models
import sounddevice
from transformers import BarkModel, BarkProcessor
import torch
import numpy as np
from optimum.bettertransformer import BetterTransformer
from scipy.io.wavfile import write as write_wav
import re
Error Message:**The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's attention_mask to obtain reliable results.
Traceback (most recent call last):
File "F:\OllamaRAG\BarkUsage\BarkUsage.py", line 56, in
barkspeed("""Hey, have you heard about this new text-to-audio model called "Bark"?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\OllamaRAG\BarkUsage\BarkUsage.py", line 47, in barkspeed
audio = model.generate(inp, do_sample=True, fine_temperature=0.4, coarse_temperature=0.5)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Program Files\anaconda3\envs\ollamaRAG\Lib\site-packages\torch\utils_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "F:\Program Files\anaconda3\envs\ollamaRAG\Lib\site-packages\transformers\models\bark\modeling_bark.py", line 1737, in generate
coarse_output = self.coarse_acoustics.generate(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Program Files\anaconda3\envs\ollamaRAG\Lib\site-packages\transformers\models\bark\modeling_bark.py", line 1078, in generate
semantic_output = torch.hstack([x_semantic_history, semantic_output])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument tensors in method wrapper_CUDA_cat)
Expected behavior
I used the code to generate some audio. Before I upgraded transformers and bark, the voice preset didn't work, bark kept changing preset. In the first half part of call function in Barkprocessor, it seemed fine, tensors were loaded properly. But in the generate function history_prompt was empty at first, then it was loaded as all 10000, After I upgraded transformers and bark, the error message shows. And after I delete the voice_preset=SPEAKER part, the code works, but with changing preset as well. Please could anyone tell me how I can get the preset to work.
LysandreJik
commented
5 days ago
System Info
transformersversion: 4.47.0.dev0Who can help?
@ylacombe
Information
Tasks
examplesfolder (such as GLUE/SQuAD, ...)Reproduction
Code: from bark import SAMPLE_RATE, generate_audio, preload_models import sounddevice from transformers import BarkModel, BarkProcessor import torch import numpy as np from optimum.bettertransformer import BetterTransformer from scipy.io.wavfile import write as write_wav import re
def barkspeed(text_prompt): processor = BarkProcessor.from_pretrained("suno/bark-small") model = BarkModel.from_pretrained("suno/bark-small", torch_dtype=torch.float16).to(device) model = BetterTransformer.transform(model, keep_original_model=False) model.enable_cpu_offload() sentences = re.split(r'[.?!]', text_prompt) pieces = [] for sentence in sentences: inp = processor(sentence.strip(), voice_preset=SPEAKER).to(device) audio = model.generate(*inp, do_sample=True, fine_temperature=0.4, coarse_temperature=0.5) audio = ((audio/torch.max(torch.abs(audio))).numpy(force=True).squeeze()pow(2, 15)).astype(np.int16) pieces.append(audio) write_wav("bark_generation.wav", SAMPLE_RATE, np.concatenate(pieces)) sounddevice.play(np.concatenate(pieces), samplerate=24000) sounddevice.wait()
Error Message: **The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's
barkspeed("""Hey, have you heard about this new text-to-audio model called "Bark"?
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\OllamaRAG\BarkUsage\BarkUsage.py", line 47, in barkspeed
audio = model.generate( inp, do_sample=True, fine_temperature=0.4, coarse_temperature=0.5)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Program Files\anaconda3\envs\ollamaRAG\Lib\site-packages\torch\utils_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "F:\Program Files\anaconda3\envs\ollamaRAG\Lib\site-packages\transformers\models\bark\modeling_bark.py", line 1737, in generate
coarse_output = self.coarse_acoustics.generate(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "F:\Program Files\anaconda3\envs\ollamaRAG\Lib\site-packages\transformers\models\bark\modeling_bark.py", line 1078, in generate
semantic_output = torch.hstack([x_semantic_history, semantic_output])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument tensors in method wrapper_CUDA_cat)
attention_maskto obtain reliable results. Traceback (most recent call last): File "F:\OllamaRAG\BarkUsage\BarkUsage.py", line 56, inExpected behavior
I used the code to generate some audio. Before I upgraded transformers and bark, the voice preset didn't work, bark kept changing preset. In the first half part of call function in Barkprocessor, it seemed fine, tensors were loaded properly. But in the generate function history_prompt was empty at first, then it was loaded as all 10000, After I upgraded transformers and bark, the error message shows. And after I delete the voice_preset=SPEAKER part, the code works, but with changing preset as well. Please could anyone tell me how I can get the preset to work.