Bug when fairseq training is ended

DeNederlandscheBank / nqm

A Transformer-based Machine for answering questions on insurance companies

MIT License

0 stars 0 forks source link

Bug when fairseq training is ended #10

Closed jm-glowienke closed 3 years ago

jm-glowienke commented 3 years ago

/Users/jan_marcglowienke/anaconda3/envs/nqm_fairseq/lib/python3.8/multiprocessing/resource_tracker.py:216: UserWarning: resource_tracker: There appear to be 3 leaked semaphore objects to clean up at shutdown\

[x] What does this mean
[x] Does it need to be fixed?
[x] Fix if necessary

jm-glowienke commented 3 years ago

This is something about exceeding the memory of the cpu. Didn't came up on the Aachen cluster up to now.

jm-glowienke commented 3 years ago

This can be easily fixed by reducing the batch-size when training the fairseq model. By this, we can ensure that the cpu is not overloaded.