alternatives to chaco/traits

fabioedoardoluigialberto commented 11 years ago

We once discussed some alternatives to chaco/traits packages. Remove dependencies from Enthought packages would mean easier, faster installations and no need for multiple python instances. Plus, Enthought documentation and support are poor.

a possible alternative for traits could be guidata: https://code.google.com/p/guidata/
a possible alternative for chaco: http://code.google.com/p/guiqwt/

What do you think?

giacomoi commented 11 years ago

Obviously, you spent some time on this already... What do you think? Let us know your opinion...

g.

On 07/11/12 14:16, fabioedoardoluigialberto wrote:

We once discussed some alternatives to chaco/traits packages. Remove dependencies from Enthought packages would mean easier, faster installations and no need for multiple python instances. Plus, Enthought documentation and support are poor.

*
a possible alternative for traits could be guidata:
https://code.google.com/p/guidata/
*
a possible alternative for chaco: http://code.google.com/p/guiqwt/
What do you think?

— Reply to this email directly or view it on GitHub https://github.com/inincs/pyNCS/issues/8.

Giacomo Indiveri http://ncs.ethz.ch/ Neuromorphic Cognitive Systems Institute of Neuroinformatics University of Zurich and ETH Zurich Winterthurerstrasse 190, 8057 Zurich, Switzerland office: Y55 G84 tel: +41 44 6353024 fax: +41 44 6353025

sheiksadique commented 11 years ago

I have tried "using" guiqwt library before. The installation is totally smooth and glitch free for Ubuntu.

What about Fedora ? Can you try and see if you can install it ? I just had to install about 114MB of additional packages when I tried installing guiqwt on my system. So if you have to install it manually, I have a feeling it might not be entirely trivial.

fabioedoardoluigialberto commented 11 years ago

I go all-in for those... :+1:

I don't think we can find better solutions for the moment. Among the pros, I found the following:

guidata has the same simplicity of Traits and it's built on Qt.
guiqwt is also built on Qt (no need for different graphical backends)
they don't have complicated package dependencies (pygtk, cairo, pango, ...)

I'll try on Fedora.

EDIT:

It works and it's beautiful. Love it.

eneftci commented 11 years ago

Hi everybody,

There was a recent similar discussion about a month ago. I've tried guiqwt back then and I was rather satisfied. Somebody is writing a module that uses guiqwt for real-time neural like plots within the neo package ( http://neuralensemble.org/trac/neo) spykeutils ( http://spykeutils.readthedocs.org/en/latest/). I've contacted the main contributor back then. Serious and responsive guy that is happy for external contribution.

guiqwt is fast but not overly fast. I attached script I wrote that gives you an idea how a raster plot might look like (must have guiqwt installed). Nevertheless I think it is the way to go.

Regards, E.

On Wed, Nov 7, 2012 at 5:41 AM, fabioedoardoluigialberto < notifications@github.com> wrote:

I go all-in for those... [image: :+1:]

I don't think we can find better solutions for the moment. Among the pros, I found the following:

-

guidata has the same simplicity of Traits and it's built on Qt.

guiqwt is also built on Qt (no need for different graphical backends)

they don't have complicated package dependencies (pygtk, cairo, pango, ...)

I'll try on Fedora.

— Reply to this email directly or view it on GitHubhttps://github.com/inincs/pyNCS/issues/8#issuecomment-10148451.

Emre Neftci, Phd. Web: http://www.ini.unizh.ch/~emre/

fabioedoardoluigialberto commented 11 years ago

Right, now I remember.

Spikeutils is nice but it would replace both the graphical and the data handling parts, basically what also NeuroTools or whatever branch we are using now is doing. Sticking to the simplest thing remains the safest option. In fact:

 "Neo is a package for representing electrophysiology data in Python"

and I remember we mentioned it in our discussions dreaming of a tool with which we could handle both spiking and analog data from the chips. It's cool but we don't need it now I suppose.

For what concerns only the graphical part, we don't need much written code, we should write it ourselves so that we know exactly what's going on, which means we should use the "mean_rate", "raster" and co. functions we already have. I don't want any more Enthought/NeuralEnsemble shocking experiences... Notice that both Neo and Spikeutils are listed as 0.2 version, which means alpha:

 "Alpha software can be unstable and could cause crashes or data loss." (Wikipedia)

:)

eneftci commented 11 years ago

Agreed. Both are GPL so we can cannibalize some code in any case. This is what I did with the demo I shared earlier :)

On Wed, Nov 7, 2012 at 9:45 AM, fabioedoardoluigialberto < notifications@github.com> wrote:

Right, now I remember.

Spikeutils is nice but it would replace both the graphical and the data handling parts, basically what also NeuroTools or whatever branch we are using now is doing. Sticking to the simplest thing remains the safest option. In fact:

"Neo is a package for representing electrophysiology data in Python"

and I remember we mentioned it in our discussions dreaming of a tool with which we could handle both spiking and analog data from the chips. It's cool but we don't need it now I suppose.

For what concerns only the graphical part, we don't need much written code, we should write it ourselves so that we know exactly what's going on, which means we should use the "mean_rate", "raster" and co. functions we already have. I don't want any more Enthought/NeuralEnsemble shocking experiences... Notice that both Neo and Spikeutils are listed as 0.2 version, which means alpha:

"Alpha software can be unstable and could cause crashes or data loss." (Wikipedia)

:)

— Reply to this email directly or view it on GitHubhttps://github.com/inincs/pyNCS/issues/8#issuecomment-10157652.

Emre Neftci, Phd. Web: http://www.ini.unizh.ch/~emre/

sheiksadique commented 11 years ago

Alright since we have an agreement, I will start off with creating a new branch "guigwt" where I will try and replace enthought stuff with guidata.

But, at this juncture, we need to consider issue #4 and see if we can have a headless version which can then use the gui stuff.

eneftci commented 11 years ago

guiqwt :)

Thanks for considering issue #4

On Wed, Nov 7, 2012 at 9:55 AM, sheiksadique notifications@github.comwrote:

Alright since we have an agreement, I will start off with creating a new branch "guigwt" where I will try and replace enthought stuff with guidata.

But, at this juncture, we need to consider issue #4https://github.com/inincs/pyNCS/issues/4and see if we can have a headless version which can then use the gui stuff.

— Reply to this email directly or view it on GitHubhttps://github.com/inincs/pyNCS/issues/8#issuecomment-10158029.

Emre Neftci, Phd. Web: http://www.ini.unizh.ch/~emre/

sheiksadique commented 11 years ago

guidata has bugs with the sliders which is more or less the only functionality we really need. I posted the issue on their page but we will have to wait and see if and when it gets fixed.

Also the plot rendering is not as fast as it was with chaco.

So.. I think we should stop bothering with this guidata/guiqwt for a bit. Any other alternatives ?

fabioedoardoluigialberto commented 8 years ago

Time for some modern visualization tools, folks. Here are some implementations I tried today, without physical address translation. I'm not sure where is the best place to put the code so I'll drop it here for reference.

_spoiler_ I guess at the moment I'm tempted to have a deeper look at the matplotlib widgets, IPython html widgets and Qt.

bokeh

~~Very fast~~ Supposed to be fast, very fancy, allows server/client architecture (to visualize data everywhere without the need for pyncs tools on the client side, just a browser. I was able to plot the events from the chip in real time but couldn't rescale the x axis, so got frustrated... (See the code for the matplotlib for how to get the events from the server, just a socket.) Here is some code. It turns out it's not very fast, and not real time... You can start it by launchin bokeh serve and then python this_code.py on a separate terminal, which will pop up a browser window. (see this guide).

#!/usr/bin/env python
#-*- coding:utf-8 -*-

import time
import socket
import sys

import numpy as np
from bokeh.client import push_session
from bokeh.plotting import figure, curdoc
from bokeh.models import Range1d

BUFSIZE = 1024 * 8

x = [time.time()]
y = [np.random.rand()]

p = figure(tools='', width=1200, height=300, title='Spikes')
s = p.scatter(x, y, color="black")

# open a session to keep our local document in sync with server
session = push_session(curdoc())

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
    sock.connect(('127.0.0.1', 50001))
except socket.error as err:
    print err
    sys.exit()

def monitor(BUFSIZE=BUFSIZE):
    received = sock.recv(BUFSIZE)
    data = np.fromstring(received,
                         dtype='uint32').reshape(-1, 2)
    # addr translation here!
    return data

tic = time.time()

samples = monitor()

delta_t = 10  # sec

def update_samples(samples):
    samples = np.row_stack((samples, monitor()))
    samples = samples[(np.max(samples[:, 0])-samples[:, 0])*1e-6 < delta_t]
    return samples

def update():
    global samples
    samples = update_samples(samples)
    s.data_source.data['x'] = samples[:, 0]
    s.data_source.data['y'] = samples[:, 1]
    s.data_source.trigger('data', s.data_source.data, s.data_source.data)
    t = time.time()
    #p.x_range = Range1d(np.max(samples[:, 0])/1e6 - delta_t,
    #                    np.max(samples[:, 0])/1e6)

curdoc().add_periodic_callback(update, 20)

session.show() # open the document in a browser

session.loop_until_closed() # run forever

matplotlib nbagg backend within the ipython/jupiter notebook

I managed to handle biases easily with things like:

def set_bias(value):
    chip.set_parameter('pinj', value)
a = interact(set_bias, I_in=(2.75, 2.9, 0.0001))

and a slider appears. Realtime plotting of streaming data using the nbagg backend requires to interact with the backend through some thread which I don't know how to and it's probably not worth the try. Maybe...

matplotlib animation tools

Very easy to implement, the fastest implementation I obtained, thought still minor issues present (e.g., window rescaling). It may be hard to get widgets in but maybe animations and matplotlib widgets can be integrated in some way, somebody must have looked at it... Here is the matplotlib example, without address translation:

import socket
import sys
import time

import numpy as np
import matplotlib.pyplot as plt
import matplotlib.animation as animation

BUFSIZE = 1024*8
DELTA_T = 2

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
    sock.connect(('127.0.0.1', 50001))
except socket.error as err:
    print err
    sys.exit()
sock.settimeout(10)

def monitor():
    # Shift data left
    try:
        received = sock.recv(BUFSIZE)
        data = np.fromstring(received, dtype='uint32').reshape(-1, 2)
        # ADDRESS TRANSLATION HERE!
    except socket.timeout:
        print 'no spikes'
        data = -np.ones((1, 2))
    return data

fig, ax = plt.subplots()

data = monitor()
try:
    tic_hw = np.min(data[:, 0])*1e-6
except:
    print "I can't start without a spikes because I can't set the time reference."
    sys.exit()

tic = time.time()

# Generate line plots
line, = ax.plot([], [], 'o', color='k', markersize=3)
ax.set_ylim((0, 128))

# Update function
def update(i, line, ax, get_data):

    global data, tic, tic_hw

    toc = time.time()

    data_new = get_data()
    # Append new values
    data = np.row_stack((data, data_new))

    toc_hw = np.max(data[:, 0])*1e-6
    data = data[(toc_hw-data[:, 0]*1e-6)<DELTA_T]

    # Update data
    line.set_data(data[:, 0]*1e-6-tic_hw, data[:, 1])

    ax.set_xlim(toc_hw-tic_hw-DELTA_T, toc_hw-tic_hw)
    ax.set_ylim((0, 128))

    if abs(toc_hw-tic_hw) > 1e4:
        tic_hw = np.min(data[:, 0])*1e-6

    return line,

ani = animation.FuncAnimation(fig, update, interval=10,
                              fargs=(line, ax, monitor), blit=False)
plt.show()

Qt4 and matplotlib

It's supposed to be the obvious way to go for standalone apps. Fast, probably as fast as the matplotlib animation, not very modern though, I still tend to prefer the fanciness of bokeh. Here is the PyQt4 code. This code works but for some weird reason the events are either sticking to the plot or the whole plot flashes, depending on how you tweak the canvas.draw() and canvas.blit() things... I guess I'm missing something.

#!/usr/bin/env python
#-*- coding:utf-8 -*-

from matplotlib.backends.backend_qt4agg import FigureCanvasQTAgg
from matplotlib.figure import Figure
import numpy as np
import socket
import time

from PyQt4 import QtGui, QtCore

BUFSIZE = 1024*8

class MatplotlibWidget(QtGui.QWidget):
    def __init__(self, parent=None):
        super(MatplotlibWidget, self).__init__(parent)

        self.figure = Figure(figsize=(20, 12))
        self.canvas = FigureCanvasQTAgg(self.figure)

        self.axis = self.figure.add_subplot(111)
        self.axis.hold(False)
        self.bkg = self.figure.canvas.copy_from_bbox(self.axis.bbox)
        self.pts = self.axis.plot([], [], 'o-', markersize=3, lw=0, color='k')[0]

        self.layoutVertical = QtGui.QVBoxLayout(self)
        self.layoutVertical.addWidget(self.canvas)

    def update_plot(self, times, neurons, delta_t):
        self.pts.set_data(times, neurons)
        self.axis.set_xlim(np.max(times[-1])-delta_t, np.max(times))
        self.axis.set_ylim((0, 128))
        self.figure.canvas.restore_region(self.bkg)
        self.axis.draw_artist(self.pts)
        self.figure.canvas.blit(self.axis.bbox)
        self.figure.canvas.draw()

class ThreadSample(QtCore.QThread):
    newSample = QtCore.pyqtSignal(np.ndarray)

    def __init__(self, parent=None):

        self.t_start = None
        self.n_iters = 0

        self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
        try:
            self.sock.connect(('127.0.0.1', 50001))
        except socket.error as err:
            print err
            sys.exit()
        super(ThreadSample, self).__init__(parent)

    def run(self):
        sample = self.monitor(BUFSIZE)
        if self.t_start is None:
            self.t_start = np.min(sample[:, 0])/1e6
        self.newSample.emit(sample)
        self.n_iters += 1

    def monitor(self, BUFSIZE):
        received = self.sock.recv(BUFSIZE)
        data = np.fromstring(received,
                             dtype='uint32').reshape(-1, 2)
        # put addr translation here!
        return data

class MyWindow(QtGui.QWidget):
    def __init__(self, parent=None, fps=25):
        super(MyWindow, self).__init__(parent)

        self.pushButtonStart = QtGui.QPushButton(self)
        self.pushButtonStart.setText("Start")
        self.pushButtonStart.clicked.connect(self.on_pushButtonStart_clicked)

        self.pushButtonStop = QtGui.QPushButton(self)
        self.pushButtonStop.setText("Stop")
        self.pushButtonStop.clicked.connect(self.on_pushButtonStop_clicked)

        self.delta_t = 5
        self.fps = fps

        self.matplotlibWidget = MatplotlibWidget(self)

        self.layoutVertical = QtGui.QVBoxLayout(self)

        self.layoutStartStop = QtGui.QHBoxLayout(self)
        self.layoutStartStop.addWidget(self.pushButtonStart)
        self.layoutStartStop.addWidget(self.pushButtonStop)

        self.layoutVertical.addLayout(self.layoutStartStop)
        self.layoutVertical.addWidget(self.matplotlibWidget)

        self.threadSample = ThreadSample(self)
        self.threadSample.newSample.connect(self.on_threadSample_newSample)
        self.threadSample.finished.connect(self.on_threadSample_finished)

        self.on = False

    @QtCore.pyqtSlot()
    def on_pushButtonStart_clicked(self):
        self.on = True
        self.tic = time.time()
        self.samples = np.zeros((1, 2))
        self.matplotlibWidget.axis.clear()
        self.threadSample.on = True
        self.threadSample.n_iters = 0
        self.threadSample.start()

    @QtCore.pyqtSlot()
    def on_pushButtonStop_clicked(self):
        self.on = False
        print self.threadSample.n_iters/(time.time()-self.tic), "fps"

    @QtCore.pyqtSlot(list)
    def on_threadSample_newSample(self, sample):
        self.samples = np.row_stack((self.samples, sample))
        times = self.samples[:, 0]*1e-6-self.threadSample.t_start
        self.samples = self.samples[times>(np.max(times)-self.delta_t)]
        #self.matplotlibWidget.axis.plot(self.samples[:, 0]*1e-6-self.threadSample.t_start,
        #                           self.samples[:, 1],
        #                           'o', color='k', markersize=3)
        self.matplotlibWidget.update_plot(self.samples[:, 0]*1e-6-self.threadSample.t_start,
                                          self.samples[:, 1],
                                          self.delta_t)

    @QtCore.pyqtSlot()
    def on_threadSample_finished(self):
        if self.on:
            time.sleep(1./self.fps)
            self.threadSample.start()

if __name__ == "__main__":
    import sys

    app = QtGui.QApplication(sys.argv)
    app.setApplicationName('Spikes')

    main = MyWindow(fps=25)
    main.resize(1024, 640)
    main.show()

    sys.exit(app.exec_())

fabioedoardoluigialberto commented 8 years ago

This is the latest version of the qt 1D visualizer. Remaining issue:

the graphics is not very smooth, I couldn't figure out why
[easyfix] if the viewer is stopped for long time the client will block the events and the setup hangs
[easyfix] no hardware translation still

qt.tar.gz

eneftci commented 8 years ago

Thanks! Very glad to see that someone is working on this. Is there anyway you can write in dummy events so we can test? Thanks.

fabioedoardoluigialberto commented 8 years ago

You can edit the spike generator I have in this script. The basic usage is (you may have to tune the argument, look at the class definition code):

>>> from poissoninp import PoissonStimulator
>>> stimulator = PoissonStimulator()
>>> simulator.rates = [[MY_PHYSICAL_ADDRESS, MY_RATE]]
>>> stimulator.start()

This will send spike trains with given rate to each specified (address, rate) pair. You may need to tune channel, host and so on... Sorry for being lazy. (By the way, a stimulator like this should also be part of PyNCS.)

poissoninp.tar.gz

eneftci commented 8 years ago

Agreed that it should be part of pyNCS. I get the error:

/home/eneftci/Projects/lib/python2.7/site-packages/NeuroTools/io.py in () 27 28 ---> 29 from pyNCS.pyST.STsl import check_dependency 30 31 import os, logging, cPickle, numpy

ImportError: cannot import name check_dependency

:(

fabioedoardoluigialberto commented 8 years ago

Sorry I don't know about that issue... It seems to be pyST so I'm not sure how that can be triggered by the my scripts. Can you give more details?

Following up on the topic, I've seen that the Manchester people use OpenGL (probably the way to go...) but their visualizer seems to be in a pretty basic form as ours is, judging by the code (I didn't run any of that).

So, I just installed PyOpenGL and will keep you updated ;)

I will follow this basic tutorial to start with.
there's probably some useful material also here
although here it's even better...
another great alternative, very straight forward

fabioedoardoluigialberto commented 8 years ago

An other update.

The great alternative is the solution I'm going after. For the aer1Dviewer there is basically the whole implementation in the examples (llok at the attachment), I just have to replace the data generation with the socket.recv thing from the aex server. For 2D viewers, I guess it's easy to go after a modification of the 1D viewer but I wouldnt be surprised to find something in the examples already.

Note that a complete re-implementation based on plain wx-python is also tempting...

wxmplot_example.tar.gz

giacomoi commented 8 years ago

Thanks Fabio! This looks very nice!

I agree that a complete re-implementation on wx-python would be ideal...

g.

Fabio Stefanini notifications@github.com writes:

An other update.

The great alternative is the solution I'm going after. For the aer1Dviewer there is basically the whole implementation in the examples (llok at the attachment), I just have to replace the data generation with the socket.recv thing from the aex server. For 2D viewers, I guess it's easy to go after a modification of the 1D viewer but I wouldnt be surprised to find something in the examples already.

Note that a complete re-implementation based on plain wx-python is also tempting...

wxmplot_example.tar.gz

You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/inincs/pyNCS/issues/8#issuecomment-207579743

Giacomo Indiveri Institute of Neuroinformatics University of Zurich and ETH Zurich Winterthurerstrasse 190 8057 Zurich, Switzerland Office: Y55 G84 TEL: +41 44 6353024 FAX: +41 44 6353025 WEB: http://ncs.ethz.ch/ http://ini.uzh.ch/

Publications: https://goo.gl/SkWujh Calendar: https://goo.gl/Dcs3Wx Appointment request: http://doodle.com/giacomoi

eneftci commented 8 years ago

I'm sold on this one (But maybe because it was the only one I managed to run).

Thanks! Emre

inincs / pyNCS