pchlap
commented
1 year ago Hi @SimonBiggs, this all sounds really exciting. Very keen to progress things on this front.
Since this is shaping up to be a research project (or sub-project) I would suggest setting up a kick-off meeting with everyone interested as well as @LoisHolloway. We'd need to break down the tasks here so that we can properly split up the tasks and make sure we have the appropriate resources to be able to do this. It's also always a good idea to agree on authorship (in particular first authorship) upfront. I think there is a lot in this so there may even be opportunities for more than one paper depending on how things progress.
So I think a meeting would be a good next step to brainstorm and start to formulate all of this. I imagine Shrikant would also be interested to be involved in this.
SimonBiggs
LoisHolloway
@rnfinnegan, that was a great talk on the work you and @pchlap have been doing in Platipy undergoing approximately anatomically relevant deformations.
As discussed at the conference I had a thought that it might be quite helpful to utilise that functionality within an autosegmentation testing suite. A deformation can be undergone, and it can be used to detect whether or not the segmentation has got worse. The difference in the segmentation quality can then be combined with https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.basinhopping.html to then search for a "worst performing deformation". This can then be used to give feedback to the model developer, in their search for failing cases. It might also be able to be a very useful metric in reporting one aspect of the generisability of a model. Essentially one could have an "Adversarial Dice" metric and an "Adversarial Hausdorff Distance" metric, and others.
If everyone was interested it would be quite nice to pull in @JakeKendrick1, Brani, and Daniel and create the implementation and write a paper together.
As a disclosure from my end, it would be very helpful for me and my product, as having a range of robust autosegmentation testing tools that I can include within my GitHub CI suite would help me make a significantly more robust product. This would also be a nice component to be able to show to the TGA through my regulatory process.
Keen to hear what the next steps in the collaboration are seen to be.