sejong-rcv / MLPD-Multi-Label-Pedestrian-Detection

[RA-L with IROS2021] Multi-Label Pedestrian Detection in Multispectral data
56 stars 13 forks source link

About the cuda memory #24

Closed medicineqian closed 7 months ago

medicineqian commented 11 months ago

As shown in the figure below, I have a memory explosion problem when running the code, there is no problem with the model training, but when running to val_epoch there will be memory explosion, that is, when running to the line of detect_object() in the inference.py there will be not enough memory. K{(AK5V@`49 U1FIQJLG${2

socome commented 11 months ago

@medicineqian

I'm not sure. It seems like the model isn't trained enough for evaluation. (That is, the trained model predicts too many bounding boxes, leading to a memory explosion) To solve the problem, try to evaluate the model after 5 epochs.

if epoch > 5 : 
     val_epoch()