Open Ted678Wu opened 3 years ago
@fsx950223 @mingxingtan could you guys give me some ideas? thanks in advance.
Export the model to saved_model Used the saved_model and run model_inspect with runmode= saved_model_infer
And outputs will be save in appropriate dir, u can see bboxs in the images saved
Export the model to saved_model Used the saved_model and run model_inspect with runmode= saved_model_infer
And outputs will be save in appropriate dir, u can see bboxs in the images saved
Hi @kartik4949 Thanks for the reply, yes, I have check the predicted image, since only around 40 images in my validation cohort, I check them one by one, and totally 20 bboxs shown on them, while the ground truth should be 150 bbox
config.yaml file and training command?
config.yaml file and training command?
training command: python main.py --mode=train_and_eval \ --training_file_pattern="C:/Workspace/1_data/3_Marker/train.tfrecord" \ --validation_file_pattern="C:/Workspace/1_data/3_Marker/val.tfrecord" \ --model_name=efficientdet-d4 \ --model_dir=tmp/model_dir/6_ab_marker_D4_1024_all/ \ --ckpt=efficientdet-d4 \ --train_batch_size=2 \ --eval_batch_size=1 --eval_samples=49 \ --num_examples_per_epoch=1088 --num_epochs=25 \ --hparams=C:/Workspace/5_EfficientDet/automl-master/efficientdet/config.yaml after training, the below config file generated.
infer command: python model_inspect.py --runmode=saved_model_infer \ --model_name=efficientdet-d4 \ --saved_model_dir=tmp/Z_models/6_ab_marker_D4_1024_all \ --output_image_dir=tmp/ \ --input_image="C:/Workspace/1_data/3_Marker/val2018/*.tif" \ --hparams=C:/Workspace/5_EfficientDet/automl-master/efficientdet/tmp/model_dir/6_ab_marker_D4_1024_all/config.yaml
`act_type: swish alpha: 0.25 anchor_scale: 0.25 apply_bn_for_resampling: true aspect_ratios:
config.yaml file and training command?
btw, any idea for the first question, my object is so small....
Increase no of epochs, may be 300
Increase no of epochs, may be 300
thanks for your idea, any idea for the inference? or do you know how to output the prediction bbox when we do the mAP evaluation?
For small objects, you should change aspect_ratios
to values get by https://github.com/google/automl/issues/412.
For small objects, you should change
aspect_ratios
to values get by #412.
Hi @fsx950223 ,thanks for the answer, I already use the k-mean to calculate the ration and try with the new ratio, but it didn't help too much, only around AP(0.5:0.95) 5% increase, do you have any other idea? thanks!
BTW: is possible to use multi-gpus training on windows? how to do that? since I met the error message about nccl...
You should try bigger batch size with grad_checkpoint
You should try bigger batch size with grad_checkpoint
@fsx950223 nice to receive your message quickly, yes, now I am using D4, training batch size 2, grad_checkingpoint True, GTX 1080Ti.
my next idea is possible to use multi-gpu on windows? since I have other GPUs available (on windows system)? in that case, I can increase my batch size.
Only batch_size 2?
Only batch_size 2?
Yes, only 2, and the GPU usage is 95% for batch size 2
Could you try bigger batch size?
I tried batch size 4 once, but the OOM message prompt up
Could you try efficientdet-d0 with batch_size 32
since all my image with size 1024, I have not tried D0 yet, maybe one option of the current situation, can I ask what's the point to try D0 with batch_size 32?
Maybe something wrong with your grad_checkpoint config.
Maybe something wrong with your grad_checkpoint config.
I see..., I will give it a try later when I can access my computer. thanks for your suggestions, and keep you updated!
Maybe something wrong with your grad_checkpoint config.
could you give me any information about multi GPU training on windows? really curious about that.
Could you try efficientdet-d0 with batch_size 32
@fsx950223 I just tried D0 with batch size 32, and grad_checkpoint: True. For such a setting, the training went well, and my GPU setting usage is around 9500MiB/11264MiB (from nvidia-smi)
Hi there, thanks for the nice script provided first. when I applied efficientdet D4 on my own dataset, I met two questions, does anyone can help me with that:
regarding 2nd question, the reason could be the preprocessing and nms? since I didn't find the preprocessing by using model_inspect?
how I can get and save the bbox prediction in 1st question, then maybe I can check the full prediction result.
thanks!