search

LumenPallidium / backprop-alts

This repository has implementations of various alternatives to backpropagation for training neural networks.
MIT License
18 stars 5 forks source link

300,000 samples in plot? But MNIST has 60k + a new backprop alt for you to try! #1

Open mi3law opened 3 months ago

mi3law commented 3 months ago

Hi LP!

I stumbled on your repo here since we're building our own alternative to backprop at aolabs.ai, using weightless neural networks.

Unless I'm missing something, MNIST has only 70k samples total (60k training, 10k testing), so I'm not sure how you have 300,000 samples in your Top1-Accuracy vs Samples plot. Did you use EMNIST?

I'd love to show you what we've cooked up applying our WNNs to MNIST, too. Seems like we're much more sample efficient. Let's chat some time!

LumenPallidium commented 3 months ago

Hello! You are correct, I was doing five epochs for each network! That does sound interesting, I'll read some of the papers on your website.

mi3law commented 1 month ago

Hi again LP!

We put up our MNIST demo app online, try it: https://mnist.aolabs.ai/ The code behind it is in this repo.

We'd love for you to run our method against the other backprop alts you've implemented already. I guess that would involve writing a script similar to the ones in this folder? We are really curious to see the performance comparisons so we can help you do this!

Can we please set up a quick chat to dig into this? Maybe Friday or sometime next week? I'd love to meet you, here's my calendar. My colleague @spolisar might join us timezones permitting (he's in Virginia, I'm currently in Kuwait).

Our approach uses weightless neural networks, so we are building with our own framework ao_core instead of torch, so you'd need access to our private repo ao_core and we can help you write an integration/test script (I'll email you our private beta license). Here's an abstract that adds more details as our docs/website is still quite bare.

LumenPallidium commented 1 month ago

Hello! I'm somewhat busy with grad school these days, but should have time to meet. Depending on level of effort, I could try and put together some comparisons. I may have some time to meet Monday. I am US Pacific Coast timezone, but based on your calendly it looks like mornings here would be good for you?

mi3law commented 1 month ago

Next week I'm actually going to ACM RecSys 24, so given that you're busy as well, can we aim for the following Monday, 21st? Morning PT will work! If you please email me ali at aolabs.ai, I'll shoot you a calendar invite which we can keep flexible.

RE the level of effort, my team and I are ready and able to help make some comparisons.

I'm excited to meet you, curious what you're studying, too.