HuwCampbell / grenade

Deep Learning in Haskell
BSD 2-Clause "Simplified" License
1.44k stars 84 forks source link

Adapt MNIST example to use the original lecun data files #97

Open jrp2014 opened 4 years ago

jrp2014 commented 4 years ago

This version downloads and uses the MNIST data files, converting the data from Word8 to Double. The runMNIST.sh script does the work.

Unfortunately, it still runs out of memory on my Ubuntu VM...

Anyway, I hope that it's of some use.

jrp2014 commented 4 years ago

The CI seems to confirm that it's not just an issue with my setup.

erikd commented 4 years ago

The Data.Semigroup import needs to be wrapped in some CPP goop.

Let me do that in a separate PR.

erikd commented 4 years ago

I am actually wondering why we don't just update the documentation.

erikd commented 4 years ago

For PR branches like this one, you really should rebase rather than merge,

jrp2014 commented 4 years ago

The CI fails because gan-mnist takes too long to generate 15 examples. mnist itself still uses too much memory to run.

erikd commented 4 years ago

Where is this this PR going? If this is ever to be merged it should be rebased against master and commits like "ci typo" need to be squashed down.

jrp2014 commented 4 years ago

Thanks @erikd. I want to find time over the w/e to kick the tyres some more:

Are you going to be able to refactor the script?

erikd commented 4 years ago

Probably best if I wait until you finish what you are working on and then assuming everything is ok, I will do done final clean up run