search

CJHMPower / Simultaneous-Traffic-Sign-Detection-and-Classification-with-RetinaNet

72 stars 28 forks source link

Validate out of memory #11

Closed ziqi-zhang closed 5 years ago

ziqi-zhang commented 5 years ago

Hi, I test validation by python test.py -m valid and I encountered cuda out of memory error. I think it's because you use the original 2048x2048 image in test and don't use any validation transforms.

Here you don't use any crop or scale transformation and can you tell me why? I encountered out of memory when I use python train.py -exp model.

I wonder how do you train and validate? Is it a problem in the code?

CJHMPower commented 5 years ago

Hi, The code you highlight is not for image size crop but for rescaling in color channel. I crop images for both training and validation before the entire training step. Code can be seen in "evaluate/crop_image.ipynb". The out of memory error is frequently encountered because this project inherently requires very large GPU memory. I used two GTX-1080Ti to train the model. Ensure you have sufficient GPU resource before training and testing.

Good luck.

ziqi-zhang commented 5 years ago

@CJHMPower Thanks for your reply! I have figured out that problem but encountered another one. After cropping train and test images, I found that I need a new annotation file because the cropped images have the different file name and bounding box position. For example the original file name is 97352.jpg and the cropped file names include 97352_4_1.jpg, 97352_4_2.jpg and so on. I saw there is a new annotation variable in crop_image.ipynb but it is not saved.

Thanks

ziqi-zhang commented 5 years ago

And I wonder how did you compute per-class precision and recall, the results of each class as the table shown in the performance section? I read the code but it seems that it only computes the final result.

Thanks

CJHMPower commented 5 years ago

Yes, the bounding box position and image file name changed after cropping. You can download the annotation file after cropping in the data link I provide. You can of course save the json file in "crop_image.ipynb".

The precision and recall for each class are computed in the "evaluate/anno_func.py/eval_annos". Just count the number of correct detection, positive false, negative false for each class. Then you can follow those formula to calculate precision and recall.

Best

ziqi-zhang commented 5 years ago

Thanks! I figured it out