adjust the evaluation intervals

[17824823008]

I want to know how to adjust the evaluation intervals.What is the meaning of this parameters?When I run this program on a hardware configuration with a memory size of 32G GTX1080ti, it will be killed by the system.

[Zengyi-Qin]

It's in hypes/kittiBox.json

"logging": { "display_iter": 500, "eval_iter": 4000, "write_iter": 800, "save_iter": 4000, "max_to_keep": 10, "image_iter": 150000 }

[17824823008]

Thanks for your reply.I reproduced the evaluation part of the experiment and did not get the visualization results of the 3D detection of the test pictures set . Is it because we did not run the evaluation code completely?How to get the visualization result of 3D image detection？

[Zengyi-Qin]

The visualization code is not included in this repo. It's just drawing boxes using the output txt files

[17824823008]

Thanks for your reply.When I run the submodules of the evaluation model, the process is killed. I want to ask if this submodule takes up a lot of memory. The type of machine is GeForce RTX 2080Ti. Is it because my machine has insufficient memory? Is there a way to solve this problem?

[e271141]

Thanks for your reply.When I run the submodules of the evaluation model, the process is killed. I want to ask if this submodule takes up a lot of memory. The type of machine is GeForce RTX 2080Ti. Is it because my machine has insufficient memory? Is there a way to solve this problem?

I encounter the same issue with the same GPU. The default batch_size is 8, which exceeds the 2080Ti limits. I changed the batch_size to 4. Also, the training takes roughly 20G+ RAM :)

[17824823008]

Thanks for your reply.I have changed the batch_size to 2.I found that by shielding the part of the evaluation model, the model could be trained properly. So I felt that there was a problem with the code of the evaluation model.

[qaazii]

encountered the same problem. i changed max_steps to 5000 after first step it starts evaluation and then my system is so busy and at the end shows killed. my system is GeForce GTX 1080i 12GB memory. i am using batch_size=1 and eval_iter=500, Anyone solved the problem?

2019-06-20 01:08:45,710 INFO Step 0/5000: loss = 0.39; lr = 1.00e-04; 0.013 sec (per Batch); 78.0 imgs/sec

2019-06-20 01:08:45,711 INFO (raw) Weight: 0.04,Boxes: 0.14,Delta: 0.04,Confidence: 0.02,Delta: 0.00,Depth: 0.04,Error: 0.57,Location: 0.06,Error: 0.77,Corner: 0.05,Error: 0.27,Refine: 0.22 2019-06-20 01:08:45,711 INFO (smooth) Weight: 0.04,Boxes: 0.14,Delta: 0.04,Confidence: 0.02,Delta: 0.00,Depth: 0.04,Error: 0.57,Location: 0.06,Error: 0.77,Corner: 0.05,Error: 0.27,Refine: 0.22 2019-06-20 01:08:46,751 INFO Running Evaluation Script. Killed

[qaazii]

Thanks for your reply.I have changed the batch_size to 2.I found that by shielding the part of the evaluation model, the model could be trained properly. So I felt that there was a problem with the code of the evaluation model.

Hi, Can you tell me what do you mean by shielding part of evaluation model?

[15900854096]

Thanks for your reply.I have changed the batch_size to 2.I found that by shielding the part of the evaluation model, the model could be trained properly. So I felt that there was a problem with the code of the evaluation model.

Hi, Can you tell me what do you mean by shielding part of evaluation model?

I guss in include/tensorvision/train.py change "if (step) % eval_iter == 0 or (step + 1) == hypes['solver']['max_steps']:" to "if 0:"