how to reduce the cuda memory?

[cst781]

maybe the batch size 2 is too small, but I cannot use two RTX 2080Ti to train this model with batch size 4

[wangtianrui]

Due to the deep complex convolution operation, this model consumes a lot of GPU memory. And DNS datasets are all 30s voice. By my test, one 30s voice needs to consumes nearly 8G of GPU memory. I set the batchsize 2 to run on two 8G GPU Core. Your rtx2080ti is 11G so cannot be put into 2 batchsize. At present, I have trained this model on 2 P100, and the batchsize is 4. It took me 8 days !

[cst781]

I see, thank for your detailed reply, Another question, how is your pesq of this model? I am very confused about the pesq of this model, since in the dns challenge two paper gives two version of orignal noisy speech's pesq

[wangtianrui]

For metrics , I use this pesq and stoi. And code is :

def get_pesq(clean, estimate, rate):
    return pesq(rate, ref=clean, deg=estimate, mode='wb')

def get_stoi(clean, estimate, rate):
    return stoi(clean, estimate, rate, extended=False)

• there are FourierTrans in pesq,so pesq is divided into broadband('wb') and narrowband('nb'),usually use broadband('wb')

• and you can read the source code about making noisy data of dns . There is a random_seed in the beginning. Different seed will cause different combination of noise and clean.

• test set choose,different selection criteria will also bring different results.

The above three points are the reasons that I can think of that may cause different evaluation results.

In fact, in my opinion, the problems that different test methods want to highlight are also different. You just need to reproduce models reasonably on your computer. And test them fairly!

hope these can help you! :)

[cst781]

Thank you, that is exactly i want to do, I use the same pesq and stoi with you, could you tell me what your result of this model like pesq is?

[wangtianrui]

PESQ:
count 2400.000000
mean 3.379963
std 0.821623
min 1.063296
25% 2.748637 50% 3.600925
75% 4.087152 max 4.509835

[cst781]

thank you, it really helped me. as for the pesq, maybe the python package get a different value from the matlab version, here is the link:https://github.com/ludlows/python-pesq/issues/3

[wangtianrui]

Wow, that's something I hadn't thought of.Thank you very much for telling me that.

[cst781]

no problem， i have emailed to you，maybe i could add your wechat

[wangtianrui]

ok.I will close this issue if no more problem