search

OlaWod / FreeVC

FreeVC: Towards High-Quality Text-Free One-Shot Voice Conversion
MIT License
602 stars 111 forks source link

Segmentation fault while training #58

Closed Likkkez closed 1 year ago

Likkkez commented 1 year ago

I noticed that every now and then I get a Segmentation Fault message in the terminal. However if i open the train log it's not there and there's no traceback message or anything. Is there any way to figure out what might be causing this? 

INFO:freevc:Train Epoch: 1515 [70%]
INFO:freevc:[2.0600645542144775, 3.001354932785034, 14.571150779724121, 20.43256950378418, 2.2390851974487305, 845200, 0.00016534851772726235]
ERROR: Unexpected segmentation fault encountered in worker.
INFO:freevc:====> Epoch: 1515
INFO:freevc:Train Epoch: 1516 [5%]

It usually happens right before the Epoch is finished and then the training continues as if nothing happened.

OlaWod commented 1 year ago

might be because your cpu cores are not many enough. decrease num_workers in dataloader may solve this.

Likkkez commented 1 year ago

Hmm I doubt that's the case, I have 5950x with 16 cores

Likkkez commented 1 year ago

Nvm I think that might be because my drive got messed up while it was training