philippjfr
commented
9 years ago That's my fault I haven't been updating the HoloViews submodule reference. Will do that tonight. As you probably discovered, the latest version of HoloViews does work.
Open mjabri opened 9 years ago
philippjfr
commented
9 years ago That's my fault I haven't been updating the HoloViews submodule reference. Will do that tonight. As you probably discovered, the latest version of HoloViews does work.
mjabri
commented
9 years ago I closed the issue as it has gone away after i updated my cloned version few days ago, and the tkgui window not opening seems to be a known issue to do with matplotlib bakend to be set to agg in rc file.
mjabri
commented
9 years ago Ok, just run into problem. It seems holoviews expects connection/projection names not have '.' in them as it seems to interpret those as separators? So '.' are not allowed in projection names. Is that correct? re there any other characters not allowed?
mjabri
commented
9 years ago Further to above, it seems this is not the only problem. Indeed holoviews has issues as cannot display the gcal_sparse.ty projections. Trace below and problem should be easily reproduced.
WARNING:root:Time: 000001.00 TopoConsole01059: (KeyError) 'Afferent'
Traceback (most recent call last):
File "/home/marwan/anaconda/lib/python2.7/lib-tk/Tkinter.py", line 1536, in call
return self.func(_args)
File "/opt/git-topographica/new/topographica/external/paramtk/paramtk/init.py", line 671, in
philippjfr
commented
9 years ago Yes, HoloViews uses periods as a special character, as far as I'm aware that's the only special character. Secondly the sparse GPU projections do not yet have the appropriate methods to visualize them, I briefly looked at it a while ago and while it seemed easy enough I haven't have time to implement it yet.
mjabri
commented
9 years ago The problem of displaying projections is not only for sparse projections, but also for normal projections. Here the trace for gcal.ty:
topo_t000000.00_c1>>> WARNING:root:Time: 000001.00 TopoConsole01059: (KeyError) 'Afferent'
Traceback (most recent call last):
File "/home/marwan/anaconda/lib/python2.7/lib-tk/Tkinter.py", line 1536, in call
return self.func(_args)
File "/opt/git-topographica/new/topographica/external/paramtk/paramtk/init.py", line 671, in
philippjfr
commented
9 years ago I'll have to look into it but I'd like to point out that the GUI interface is no longer officially supported. We recommend working directly in an IPython/Jupyter notebook. You can have a look at what that looks like in practice on our Tutorials page, specifically it would look something like this for GCAL.
philippjfr
commented
9 years ago I just tested it and displaying CFs and projection works fine for me. Could you give a step by step account of how you got this error?
mjabri
commented
9 years ago I forked topographica and all sub-modules on Sep 7 (topographica --version gives me: 0.9.8-1547-g76d6eeb-dirty) I cloned the forks and sub-modules to my local linux box. I run topographica -g gcal.ty First thing i noticed: I cannot display activities as before immediately after starting. I need to run at least 1.0 before the GUI opens Activity window. Otherwise I get the message: "No suitable objects in this simulation for this operation". I run 1.0. Now I can display Activities. I try to plot projections (Plots | Projection) and I get the KeyError for 'Afferent'.
jbednar
commented
9 years ago You should be able to plot things after running for 0.0, if you want to see the initial state. There are indeed no suitable objects to plot until the simulation has been run, even for zero time.
Jim
On Wed, Sep 9, 2015 at 10:43 AM, M. Jabri notifications@github.com wrote:
I forked topographica and all sub-modules on Sep 7 (topographica --version gives me: 0.9.8-1547-g76d6eeb-dirty) I cloned the forks and sub-modules to my local linux box. I run topographica -g gcal.ty First thing i noticed: I cannot display activities as before immediately after starting. I need to run at least 1.0 before the GUI opens Activity window. Otherwise I get the message: "No suitable objects in this simulation for this operation". I run 1.0. Now I can display Activities. I try to plot projections (Plots | Projection) and I get the KeyError for 'Afferent'.
— Reply to this email directly or view it on GitHub https://github.com/ioam/topographica/issues/629#issuecomment-138951314.
mjabri
commented
9 years ago Cannot display Activities even i if i run 0.0. This issue does not bother as much as the not being able to plot projections at any time. I was just pointing this in case it is not expected, and as it is new behaviour since earlier version i am running.
jbednar
commented
9 years ago Hmm; that should have worked. I don't have it handy to test locally right now, unfortunately.
On Wed, Sep 9, 2015 at 11:01 AM, M. Jabri notifications@github.com wrote:
Cannot display Activities even i if i run 0.0. This issue does not bother as much as the not being able to plot projections at any time. I was just pointing this in case it is not expected, and as it is new behaviour since earlier version i am running.
— Reply to this email directly or view it on GitHub https://github.com/ioam/topographica/issues/629#issuecomment-138956913.
philippjfr
commented
9 years ago As I said above, I tried exactly that and it worked fine for me. Not sure what's going on here.
mjabri
commented
9 years ago Ok, i am going to clone directly from git now and try again...
philippjfr
commented
9 years ago Nothing really changed since September 7th so I don't think that'll fix it. Make sure that you update all the submodules manually, particularly featuremapper and holoviews.
mjabri
commented
9 years ago What do you mean by "manually"? I am doing "git submodule update --init" in a fresh clone in the topographica directory. Anything else I could do?
philippjfr
commented
9 years ago Nevermind, just pull again and do another git submodule update, I've just updated the references so that should get you the latest versions now.
mjabri
commented
9 years ago Still get the same problem...
philippjfr
commented
9 years ago Yea didn't think that would be the issue. No idea how to debug this as it works fine for me. Maybe @jbednar can try it later. I would strongly suggest working in the notebook even if we figure this out.
mjabri
commented
9 years ago Thanks Philipp. I am at loss with what the issue is. My anaconda installation works well with the older Topographica version I am using. Notebooks are great, and i use them in development, but my model is already instrumented in the GUI. Also, reason for me to look at latest release was the GPU implementation (the sparse CFs), but if there is no benefit (which i posted about), it is huge amount of work to shift to notebook just for the sake of this. I already connected my model to Caffe (my whole first layer is now running on GPU), and wanted to speed up the upper layers which are Settling sheets.
philippjfr
commented
9 years ago I'll have a go at it on some different systems. Totally get that you've been using the GUI for everything and it may be too late to switch so hopefully we can sort this out quickly.
As for the GPU stuff, very cool to hear you managed to interface the model with Caffe, if you could share some of that code it might be helpful as I run some of my larger models. In terms of the sparse GPU implementation there's two things to note. First of all it's meant for models that are actually sparse, while it'll match the CPU version in speed, you might only get significant gains if your CFs contain lots of zero values. As part of some of my work I tried this and found that generally you can sparsify afferent CFs to about 20-30% with very little effect on map development and for some very large lateral connections I often go to >90% sparsity. Note that when I say sparsity I mean the sparsity just in the non-situated disk of weights, since the GPU implementation is already highly sparse as it's stored as a sparse array of all possible connections. Secondly, the main purpose for the GPU implementation was for larger models, so while you won't see much gain in standard GCAL, you should see significant gains for higher area or density values or large CFs.
Over the next couple of weeks I'll be rerunning some of my larger models and will hopefully use the implementation myself and report back with some benchmarks. If you're still interested at that point we can maybe collaborate on improving the API a little bit, particularly to view the weights more easily.
If you have time it would be great if you could run the GPU benchmarking notebook and report back on the results.
mjabri
commented
9 years ago Thanks Philipp. How zeros in CF is taken advantage by GPU? Why not by CPU? The big advantage i have seen from putting my fixed sheets (sheets with pre-fixed CFs/kernels) in GPU is through the operations becoming a convolution with many prefixed kernels, and by im2cols this becoming a matrix multiplication (gemm) which is a sweet spot for GPU. What am I missing?
Also, I looked at the GPU benchmarking notebook, and it was fairly complicated to decipher! After trivial changes to fit my context (e.g. host name) I am getting a " UndefinedVariableError: name 'version' is not defined" in the Bars() statement below.
%%opts Bars [color_by=['stack'] apply_databounds=False show_grid=True] versions, densities, parts = ["Traditional_topographica", "Sparse_topographica"], [48, 142, 162], ['Topographica startup', 'Simulation startup','Simulation runtime']
keys = product(versions, densities, parts)
Bars([(k, constituents_traditional_vs_sparse.query('version == "%s" and cortex_density == %d' % (k[0], k[1]))[k[2]].mean()) for k in keys], key_dimensions=['Version', 'Cortex density', Dimension('Part', values=parts)], value_dimensions=['Time (seconds)'])
It seems the version names (["Traditional_topographica", "Sparse_topographica"]) are different from what is being generated in the script which is using git tag to get version names. Unless the version names are created somewhere else i am missing.
So this script requires quite a bit of work to make it work in other environments that it was developed for it seems.
I cloned from git on July 14th and then did git submodule update --init and got all down, and then tried to run examples/gcal.ty and i am getting holoviews import errors which even when i fixed as below, the GUI seems to be blocked. Here what the diffs look like.
< from holoviews.plotting import NdLayout, CurvePlot
< from holoviews.plotting.element import OverlayPlot
Is there any other installation step i need to do?