search

chainer / chainercv

ChainerCV: a Library for Deep Learning in Computer Vision
MIT License
1.48k stars 304 forks source link

Accuracy problems of FCIS example #915

Closed otsukaoly closed 5 years ago

otsukaoly commented 5 years ago

I'm executing FCIS sample into particle images. (See #912.)

There are additional problems in accuracy.

  1. If there are many particles in the image, number of detected particles is lesser than number of true particles obviously. Any hyperparameter may be related, but I don't know which hyperparameter is related.

  2. If use_preset method of FCIS class is called using not 'visualize' but 'evaluate', the number of detected particles is increased. (It seems that the method changes self.score_thresh variable from 0.7 to 1e-3.) However, additional detected particles are represented using the same pattern. It may be bug.

I appreciate any comments. If there is suitable site to ask accuracy of FCIS, advise it.

knorth55 commented 5 years ago

If there are many particles in the image, number of detected particles is lesser than number of true particles obviously. Any hyperparameter may be related, but I don't know which hyperparameter is related.

Probably, IoU threshold is related.

If use_preset method of FCIS class is called using not 'visualize' but 'evaluate', the number of detected particles is increased. (It seems that the method changes self.score_thresh variable from 0.7 to 1e-3.) However, additional detected particles are represented using the same pattern. It may be bug.

What do you mean by same pattern? can you upload visualized image of your result?

otsukaoly commented 5 years ago

Probably, IoU threshold is related.

Could you explain in detail which parameter you mean in the source code?

What do you mean by same pattern? can you upload visualized image of your result?

I can only upload true images and predicted images.

Case 1. There are not many particles in the image. not_many_particles

Case 2-A. In case of use_preset('visualize'). result_visualize

Case 2-B. In case of use_preset('evaluate'). Additional detected particles are represented using the same pattern. result_evaluate

otsukaoly commented 5 years ago

I uploaded true images and predicted images. If possible, I need any advice.

knorth55 commented 5 years ago

I can say nothing about your case because there is no input image. It seems that training is not done properly, so it is better to check your training log.

otsukaoly commented 5 years ago

sorry for my late reply. I adjusted some of parameters below and resolved 2 problems finally. Thank you.

FCISResNet101
    RegionProposalNetwork
        ratios and anchor_scales
    ProposalCreator
        nms_thresh, n_train_pre_nms, n_train_post_nms, n_test_pre_nms, n_test_post_nms and min_size
    FCISResNet101Head
        n_class, roi_size, group_size, loc_normalize_mean, loc_normalize_std and iter2
    nms_thresh, score_thresh, mask_merge_thresh, binary_thresh, limit and min_drop_size

FCISTrainChain
    rpn_sigma and roi_sigma
    AnchorTargetCreator
        n_sample, pos_iou_thresh, neg_iou_thresh and pos_ratio
    ProposalTargetCreator
        n_sample, pos_ratio, pos_iou_thresh, neg_iou_thresh_hi, neg_iou_thresh_lo and binary_thresh