unlockNN module for uncertainty quantification of MEGNet

[sgbaird]

Did a quick search, and it seems like some additional dropout functionality related to quantifying model uncertainty was added, but I don't think that was ever fleshed out into top-level UQ functionality with MEGNet (please correct me if I'm wrong). I came across unlockNN (unaffiliated) and thought it would be worth bringing attention to in the repo. Has some specific functionality compatible with MEGNet. Maybe worth incorporating the functionality into MEGNet if you read the paper and like the results (I haven't read it yet).

[chc273]

@sgbaird https://github.com/openjournals/joss-papers/blob/joss.03700/joss.03700/10.21105.joss.03700.pdf I think the JOSS paper is here, not sure if they have another more detailed paper planned (really looking forward to it!).

I think the idea is fascinating and I am sure @a-ws-m has done great work on unlockNN. Regarding incorporating it into megnet, I have no issue about it. It just makes me think why not we just use unlockNN directly?

[sgbaird]

@chc273 thanks for the quick response! The main thought behind the suggestion was for the uncertainty quantification to be housed directly within the MEGNet API for both visibility and ease of use. Maybe it would be easier to do this directly with unlockNN as you mentioned. The documentation site for unlockNN is down, but once it's back up I think it should be clearer what the workflow would look like (this could be as simple as some extra instructions in the MEGNet documentation). Obviously up to you and the other MEGNet devs.

[a-ws-m]

Hi both, thanks for the interest :smile:. the main blocker for unlockNN's JOSS publication is some benchmarking, which is in progress, and the plan is to write another more detailed one afterwards. Thanks also for pointing out the broken link, I need to do some merging; the docs are available here. I used MEGNet extensively during its development and there's specific documentation on using it with unlockNN here.

[chc273]

@sgbaird Thanks! My thoughts are that the unlockNN method can be used on more than just megnet, and it should not limit to megnet also. It is easier to keep them separate if someone develops a new model that can work with unlockNN.

@a-ws-m thanks for the example! Do you have a more detailed paper explaining the mechanisms behind the code and method? (not relevant to this Issue, but a personal request :) )

[a-ws-m]

Sorry for the slow reply @chc273 -- it's my intention to write a couple of papers related to it: the JOSS paper, which is nearly finished, and a more detailed one a bit further down the line. There are two papers/works in particular that this work builds upon, which have more detail than the JOSS paper at the moment: Variational Gaussian Processes (explanation in TensorFlow docs here and paper here) and Graph-fed Gaussian Processes (see here).