search

ctn-archive / nengo_theano

ABANDONED; see https://github.com/nengo/nengo instead
MIT License
3 stars 3 forks source link

Improve decoder cache #20

Open tcstewar opened 11 years ago

tcstewar commented 11 years ago

Right now, a file cache_gamma_inv is made and all A and inv(Gamma) matricies are dumped into it. The following improvements would be nice:

1) Don't use the cache if a random seed is not specified. This is consistent with Nengo's approach, and means that casual users won't ever have the weird giant file growing out of control on their harddrives

2) Clear old items from the cache. Maybe have some file size limit, or an age limit? I'm not sure how this one would work. Maybe there's an existing python module that does something like this.

3) Find out if it'd be better to split the cache into separate files in a directory, as in Nengo

tbekolay commented 11 years ago

I think a good way to do 2) is to put cache files in a temp directory, so the OS can handle clearing it when it wants. Python does have a way to handle tempfiles. I changed cache.py to do tempfiles in this commit on my fork. Let me know what you think of it -- I think it probably makes more sense to do the tempfile thing once the cache is split into separate files.

studywolf commented 11 years ago

I support the rewrite! Putting things in the tempfile folder seems like the best way to go, and rewriting to be formatted properly is nice too! If there are no objections lets get that merged in!

Also, it would be nice to have a print out statement that lets the user know any time a quickfile is being used, to avoid the case where I have no idea why something is not reflecting changes I'm making and it ends up a quickfile is being secretly called up.

studywolf commented 11 years ago

Also the cache should be a singleton factory, or whatever the term is, instead of using any global variables.

tcstewar commented 11 years ago

I like the cache in the temp directory thing. Merge it in!

One of the original reasons the Nengo quick files were all separate is that it's possible to send people the quick files if their computers are too slow to generate them on their own. I do not think this is useful for the Theano version, though.

tcstewar commented 11 years ago

Hmm, @studywolf, what are the situations where the cache is screwing up and not reflecting changes? There shouldn't be any at all -- we want the cache to not affect the actual behaviour in any way.

studywolf commented 11 years ago

It was just a couple things that were fixed, more an in general thing that if something is changed and things start going wrong we don't want people to not know that quickfiles are being used.

tbekolay commented 11 years ago

It might be good to have a DEBUG mode on all this nef-py stuff, and disable the cache when in DEBUG mode.

tbekolay commented 11 years ago

I pulled in the cache improvements from my fork.