SeuTao
commented
4 years ago Thank you!
The RNN part of this solution actually did two things: 1) sequence modeling；2) multi model(CNN) ensemble. We can just train a single 2d CNN classifier along with RNN and still get good results.
I don't think end-to-end training of multi CNN backbones is a good idea. I suggest that we train single CNN+RNN model end2end and perform ensemble learning afterwards.
Tuning the "overall" loss must be very tricky to achieve better result than the two stage solution but still worth a try.
Sorry that I'm busy with other projects the in the last few months. I'm considering continue this project and will update soon.
The work is awesome for ICH subtype detections. Exciting to see the combination of CNN and RNN networks for classification purposes.
The weakness of this model, however, requires multiple training over different CNN networks for image encoding for RNN classification usage.
I have see on the to-do list you are suggesting to have an end-to-end training, I am considering if GPU memory is sufficient for multiple CNN training using backbones likes ResNet-101.
Also would you consider to have an "overall" loss by combining CNN and RNN classification losses like the adversarial loss used in GAN? I am not sure if this can help to trim down the network size so that whole framework can train with lower GPU memory requirements.