GPU requirement for sv-eres2netv2

modelscope / 3D-Speaker

A Repository for Single- and Multi-modal Speaker Verification, Speaker Recognition and Speaker Diarization

Apache License 2.0

1.02k stars 89 forks source link

GPU requirement for sv-eres2netv2 #122

Closed Tortoise17 closed 1 month ago

Tortoise17 commented 1 month ago

How much GPU memory is requirement for training sv-eres2netv2model? I tried with 48 GB but failed. Is there any way to utilize the multi GPUs to avoid memory allocation failure?

yfchenlucky commented 1 month ago

I'm using 4 32G V100 GPUs, and 26G of memory per card in use, which is sufficient. Is your batch size too large? You could try running with the standard configuration.

Tortoise17 commented 1 month ago

@yfchenlucky I have tried run.sh file. so far I have not changed batch size. But I have used 48 G 2X cards, didn't try more than this.

yfchenlucky commented 1 month ago

You can change the batch_size to 128 or lower in https://github.com/modelscope/3D-Speaker/blob/main/egs/voxceleb/sv-eres2netv2/conf/eres2netv2.yaml#L24 and try again.