search

ctn-archive / nengo_theano

ABANDONED; see https://github.com/nengo/nengo instead
MIT License
3 stars 3 forks source link

Saving "quick" files #2

Closed studywolf closed 11 years ago

studywolf commented 11 years ago

I don't think this can be the same as in Nengo, since populations are all computed together at the end. So then this would be for saving a generated theano graph?

tcstewar commented 11 years ago

I was thinking more that this would be for saving decoders so they can be re-used, rather than computing them all the time. Same idea as in Nengo, and it's just meant to avoid the start-up time where all the matrices have to be inverted to solve for the decoders.

In Nengo, we actually just save the A matrix and the GammaInv matrix. So, when we create an ensemble that has exactly the same parameters (including seed), instead of re-building GammaInv, we just use the one from the quick file. Now when we want to generate an Origin, we just make Upsilon with dot(A, f(X)), and do decoder=dot(GammaInv, Upsilon).

Unfortunately, this approach to quick files only works for that particular way of solving for decoders. The gradient descent methods don't have a GammaInv, and neither do a lot of the sparsification methods we've been looking at. So at the moment, quick files don't help with those.

So, there's two approaches available here. One is to do what Nengo does and store the A and GammaInv matrices. The other is to be a little fancier and directly store the decoders (saving a few dot products in the normal case, and allowing other methods to be used eventually).

My vote is to just do the normal Nengo case to start with, but eventually both Nengo and this code should be made a bit more capable about this. Unfortunately, as far as I can tell there's very little implementation overlap between the Nengo and Theano versions here (except perhaps for the code that figures out the filename of the quickfile).

tcstewar commented 11 years ago

The system now creates a cache_gamma_inv file and stores the A and inv(Gamma) matrices in there. The cache works the same as the one in Nengo, except it's all in one giant file.

This should work for now, but we might consider splitting the cache up to separate files (which may help access times as it gets larger -- I'm not sure, as right now I'm just using shelve to handle the cache).

tcstewar commented 11 years ago

We should also have some way of stopping the cache from growing infinitely large. Right now, it just keeps adding to that cache file.

tcstewar commented 11 years ago

Closed, since the basic implementation is now in. Moving discussion on improvements to Issue #20