search

KovenYu / MAR

Pytorch code for our CVPR'19 (oral) work: Unsupervised person re-identification by soft multilabel learning
https://kovenyu.com/publication/2019-cvpr-mar/
315 stars 83 forks source link

MemoryError #9

Closed haofeng0928 closed 5 years ago

haofeng0928 commented 5 years ago

i dont have 40G cpu memory, is there another solution?

KovenYu commented 5 years ago

@haofeng0928 In my implementation I load the entire dataset into RAM for higher speed, but it costs more cpu momery, especially for the MSMT17. To reduce the cpu memory usage you can change the data loading style.

Specifically, you need to use the ImageFolder Dataset Constructor for the MSMT17 dataset. Please modify L521 in utils.py to use the ImageFolder. Note that source_data is a Dataset object. Once you get this done, the memory usage would be drastically reduced to about only 5GB or less.