The PrototypeContext mixin has been augmented so that now it can do either of the following during training:
Create prototype contexts via task labels while training (just as before),
Construct prototype contexts via a clustering technique in an online way while training without need for tasks labels (new!).
There is also a corresponding config file for experiments that construct the prototype context while training. A few additional things to note:
The attribute contexts (in the mixin) is a List containing a single Tensor if constructing the prototype context since lists are mutable, and we require contexts to be mutable since it is updated regularly by other functions,
The clustering procedure requires float64, so the training process itself is slightly slower when constructing prototype contexts,
There isn't any formal documentation of the exact clustering procedure in the mixin itself, and we can redirect to arXiv/bioRxiv papers.
Here are results from a stand-alone script that constructed prototypes (for continual learning on permutedMNIST) vs this Vernon implementation. The results are very similar, and averaged over 8 independent trials.
The
PrototypeContextmixin has been augmented so that now it can do either of the following during training:There is also a corresponding config file for experiments that construct the prototype context while training. A few additional things to note:
contexts(in the mixin) is a List containing a single Tensor if constructing the prototype context since lists are mutable, and we requirecontextsto be mutable since it is updated regularly by other functions,Here are results from a stand-alone script that constructed prototypes (for continual learning on permutedMNIST) vs this Vernon implementation. The results are very similar, and averaged over 8 independent trials.