Closed ZFTurbo closed 11 months ago
@ZFTurbo i'm using 2 as the batch size, with an implicit channel dimension of 1
Is it possible to remade for stereo input?
@ZFTurbo ah i see
in stereo, the channel dimension would be 2?
@ZFTurbo ah i see
in stereo, the channel dimension would be 2?
Yes )
ok, so i just do stft on each dimension and concat? sorry but i'm actually not a domain expert (besides the transformer part)
I think here is the simple example how it can be done: https://github.com/kuielab/sdx23/blob/mdx_AB/my_submission/src/tfc_tdf_v3.py
@ZFTurbo ahh got it, yea, like i guessed
yup, i can do this, and i guess the multi-stft loss can also be done per channel dimension and just summed
tada
try 0.1.0
import torch
from bs_roformer import BSRoformer
model = BSRoformer(
dim = 512,
depth = 12,
stereo = True,
time_transformer_depth = 1,
freq_transformer_depth = 1,
freqs_per_bands = (512, 513) # in the paper, they divide into ~60 bands, test with 1 for starters
)
x = torch.randn(2, 2, 131680)
target = torch.randn(2, 2, 131680)
loss = model(x, target = target)
loss.backward()
# after much training
out = model(x)
ok, calling it a day, happy friday!
Thank you. It works for me now.
One more proposal. For MSS task you sometimes need more than one stems on output.
so target can have shape like (BS, Stems number, 2, length)
target = torch.randn(2, 4, 2, 131680)
May be it's good idea to add to support for example 4 stems separation.
ok, very last commit for the day
import torch
from bs_roformer import BSRoformer
model = BSRoformer(
dim = 512,
depth = 12,
stereo = True,
num_stems = 2,
time_transformer_depth = 1,
freq_transformer_depth = 1,
freqs_per_bands = (512, 513) # in the paper, they divide into ~60 bands, test with 1 for starters
)
x = torch.randn(2, 2, 131680)
target = torch.randn(2, 2, 2, 131680)
loss = model(x, target = target)
loss.backward()
# after much training
out = model(x)
if that doesn't look right, i can address it next Monday
Thank you I will test it!
@ZFTurbo if stems is just different targets, you could also just train 4 separate models (in your example), or am i misunderstanding?
@ZFTurbo if stems is just different targets, you could also just train 4 separate models (in your example), or am i misunderstanding?
Yes, you are right. But sometimes it's better to have all in single model.
ohh got it, so 'stems' just another synonym for adapter heads
yea, this should work then
i guess you could also have a separate axial transformer block per stem head, but can leave that for another day
main issue has been addressed
Hello, there is a problem with input data. You write
But in reality there must be also some batch size:
If I use like in bottom example I have a problem with stft.