[Replication] Scaling "Topographic VAEs learn Equivariant Capsules"

[StellaAthena]

Background

A recent interesting paper in topological deep learning unfortunately only experiments on MNIST. Let’s try it on something more interesting.

Topographic VAEs learn Equivariant Capsules ArXiV abstract: https://arxiv.org/abs/2109.01394 Yannic's video: https://www.youtube.com/watch?v=pBau7umFhjQ Code: https://github.com/AKAndyKeller/TopographicVAE

What to Replicate?

It should be the same amount of work to replicate the entire paper as a subset of it I imagine?

Modifications

How about CIFAR-10 and CIFAR-100 for starters, maybe ImageNet if the initial results are good.

Related Papers/Frameworks

n/a

[excitingtimes]

Would you see any other applications of the same idea outside the field of vision for instance ?

[StellaAthena]

Would you see any other applications of the same idea outside the field of vision for instance ?

Definitely! I'm actually very interested in the idea of using Temporal Coherence (4.5.1) for time series analysis and potentially story-telling models (see my prior work on storytelling with @LouisCastricato here, here, and here)

[LouisCastricato]

diffusion models for storytelling are extremely promising- especially now since we can control the iterative refinement with CARP.

[LouisCastricato]

@excitingtimes what is your discord username.

[excitingtimes]

@LouisCastricato here's my discord --> rohan42#3508 :)

[StellaAthena]

@excitingtimes should I put you down as working on this?

[excitingtimes]

Not yet @StellaAthena as this is not my primary research interest as of now, but will definitely ping you on this chan once I start working on it :)

[hmen97]

@StellaAthena I would like to work on this, here's my fork, made a few changes

[StellaAthena]

@hmen97 Awesome! Go ham :) I’m quite interested in seeing how it goes. DM me on Discord to talk about compute resources.

[jackransomlovell]

hey, @hmen97 @StellaAthena I'd be psyched to work on this. I've focused on the VAE from a cognitive science perspective for a bit. gonna read the paper, watch yannick's vid b/w now and Monday, and review the fork b/w now and Monday. lmk if anyone wants to chat.

[imr555]

@StellaAthena , Is anyone working on this? I can go through the material and try to get the code up and running for cifar-10, and cifar-100.

If we are ambitious enough, we can try out cub-200, oxford flowers, standard cars dataset too. These datasets are known for fine grained image classification where the classes are hard to distinguish. More details here. (https://paperswithcode.com/task/fine-grained-image-classification)

[LouisCastricato]

No one is working on this.

[imr555]

@LouisCastricato , So is this abandoned, Just curious? Might try my hand at it if it's not abandoned.

[StellaAthena]

@imr555 You're quite welcome to work on it. I would need to refresh to get back up to speed but I think this is still a worthwhile project.

[imr555]

No worries, Thank You, @StellaAthena ...... Reason for the interest is, I had a brief stint with Capsules and VAEs back in 2020. I will look into it.