ENH: Python 3 compatibility and changing backend

[larsoner]

Over at mne-python we're looking into migrating to Python3, and one of our biggest roadblocks will be PySurfer / Mayavi. @agramfort has mentioned that Mayavi may not ever be migrated to 3.0, so it might be worth switching to a different backend. I also had to do some pretty ugly hacks to get TraitsUI to support visualization scripting, since it's not built to do that (it is not supported by the devs).

I'm up for looking into the various other backend options here if people are open to the idea of switching backends for 0.5. Having done a little bit of low-level OpenGL work recently, I think it might be possible to go to a backend that only gives us low-level OpenGL calls (as opposed to high-level abstraction like VTK/Mayavi) without too much hassle.

What do people think?

Here are a list of potential ones (I'll update as people suggest or I find them):

• pyqtgraph (MIT)
  • Abstractions for 3D surface plotting
  • Built-in mouse / keyboard interaction
  • Has widgets (GUI-like functionality possible)
  • Requires QT4 / PySide (Mayavi more or less did, too)
• vispy (BSD)
  • Fairly low-level access to OpenGL
• VTK no 3.0 date yet
• XTK most programming in JavaScript, Web interface makes local file usage problematic
• pyopengl / glumpy / pyglet / pygame raw OpenGL only as last resort
• VPython a distribution more than a package
• visvis (BSD) probably not complete enough mesh rendering options

[agramfort]

I am always skeptical but you proved me wrong many times so...

but please look at what's around before coding OpenGL from scratch.

[larsoner]

I've added a few potential backends to the issue description, I'll update them soon with advantages/disadvantages.

[dengemann]

At the risk of repeating myself, e.g. http://vispy.org/ The other options would be pygame and pyglet (used by pscyhopy).

[larsoner]

Looking into it, PyQTGraph might actually have everything we need. They even have widgets for interactive use. @agramfort we're okay using MIT-licensed software, right? @dengemann it looks like vispy does not have high-level functions like mesh or surf, so PyQTGraph would probably be easier to use.

[agramfort]

@agramfort we're okay using MIT-licensed software, right?

yes

[dengemann]

@Eric89GXL taking a glance at PyQTGraph it seems we can do many things with it ;-) It could also help implementing the features discussed in https://github.com/mne-tools/mne-python/pull/672 We should check how availability looks like across systems.

[dengemann]

(there is partial support for python 3; not all features work yet) from the PyQTGraph webpage. We need to check whether partial is sufficient for us.

[larsoner]

From reading through the mailing lists, it looks like they've fixed a number of issues with Python 3 this summer. Maybe I'll just try implementing it and see how it does :)

[dengemann]

@Eric89GXL the easiest way to gain some confidence would probably be to run a few basic examples from Py3k and see whether it's shaky or not. If it's solid +1 for PyQTGraph. I hope it will allow to create surfaces rendering as smoothly and nicely as with Mayavi ...

[dengemann]

Btw. happy Thanksgiving! :-)

[maedoc]

For an example of using PyQtGraph, take a look at

https://github.com/mmwoodman/inspy/blob/master/ins/ui/tvb_film.py

which visualizes a cortical surface, two forward solutions, and a controller to manipulate time.

For surface visualizations, the interesting part is the SurfaceTime.setup_surface method.

[larsoner]

That's great. Thanks for the link. Looking into it a little bit, I don't think it will be too difficult to switch, since they already have some surface code.

In terms of implementation, I think it might be nice if we make it so that users could embed PySurfer as a QT widget in an application if they wanted. pyqtgraph already makes all its plotting areas as QT widgets, so we should be able to extend its functionality to allow for all the nice PySurfer handling. I'll know more about the feasibility of this as I go to implement, but I'm planning to try to go in that direction if possible. Then the PySurfer Brain class would basically be responsible for setting up a QT window, and adding something like a QTBrain instance to the window.

@agramfort @mwaskom are we concerned about backward compatibility with the figure parameter to Brain? We /could/ deprecate and remove that functionality, but it would essentially require keeping around and testing two codebases (mayavi/TraitsUI, and the new pyqtgraph). I'd be okay with telling users to keep using 0.4 if they need Mayavi.

[agramfort]

my only concern is about our ability with pyqtgraph to have the same quality of plots. Let's see how far you go with this. But if it starts to be complex with > 2k lines of code please let's discuss our options.

[mwaskom]

This is pretty cool! I think Py3 support would be great to see.

As Alex says, if moving to PyQTGraph is doable with reasonable code seems feasible, then I think it would be a good idea.

Eric, do you know if PyQTGraph can plot in headless mode? That's pretty high on my list of desiderata for a new backend. But I guess it might not be possible for very low-level constraints.

I'm not too sure how widely used the figure parameter is. I think @satra uses it for some things?

It's worth pointing out that if we're going to to a lot of heavy lifting on the backend, it would be great to leave us in a position where we could easily integrate a browser-based backend. The xtk project looks like it's maturing nicely, and the interactive IPython javascript stuff it slated for debut in the 2.0 release soon. It would be killer to be able to use PySurfer directly in an IPython notebook!

[larsoner]

Although I'm not crazy about programming in JavaScript, XTK does look cool. The biggest limitation for me is that it actually seems difficult to access local files when working with it, since browsers intentionally make that difficult for security reasons. Maybe I'm not doing it correctly, though... In any case, it I also recall that it was missing a few features we'd need when I evaluated it as a potential backend 6 months to a year ago.

Headless mode would indeed be nice. I wonder if some of these packages would allow for forcing software rendering in certain contexts, which might be what is needed for such support. Not 100% sure though. My guess is that something like PyQTGraph is more likely than XTK to have this sort of support, but I don't really know. I could ask them how feasible that would be.

[mwaskom]

Yeah I can't imagine XTK is going to work well as a backend for distributed cluster execution (which is half of where I'd really like to be able to use PySurfer). I guess ideally we'd have a fully modular backend system, which might be a huge hassle.

[larsoner]

Some of the refactoring I did before would make it easier to have a modular backend system. I can try to continue moving in that direction when replacing Mayavi with PyQTGraph, but ultimately supporting more than one backend is likely to be quite difficult.

I'm not sure how possible it is to do OpenGL graphics on a headless system. My guess is that it should be possible, since OpenGL usually has software rendering available at some level. If you want to look into it and see what's required, we could interface with the PyQTGraph people to see if it could be done. For PyQTGraph in particular, we'd also need to know how QT widgets/windows/applications work in headless mode -- my guess is that they don't really work :) However, I think PyQTGraph uses PyOpenGL under the hood, which means at some level we might be able to hook into raw OpenGL calls if we must bypass the QT widget/window/application layer.

[aestrivex]

For what its worth -- I don't think its likely that "mayavi will never move to python3." It could take a while though.

[mwaskom]

True, but frankly mayavi development seems a little stalled out. It looks like the most recent commit is 4 months old. Don't get me wrong, mayavi has served us very well, but now that we have a much better idea of what we need than I did 3 years ago when I started hacking this thing together, it makes a lot of sense to move in a direction where PySurfer is talking more directly to the lower-level stuff anyway.

[larsoner]

Mayavi also has a limitation in terms of multiple view support when combined with scripting -- I contacted the devs about it, and they were not interested in changing the behavior.

[aestrivex]

I have a very small pitch to make for keeping pysurfer on mayavi, which is that I was thinking about eventually integrating some of its functionality into my program, cvu, specifically in a mode to overlay abstract connectivity with vertex-based activation. However, if pysurfer does move off to pyqtgraph, it wouldn't be challenging to code it myself.

[larsoner]

It turns out pyqtgraph has stopped development of their 3D plotting in favor of vispy -- they plan to eventually replace their backend with it. Looks like I should have listened to @dengemann :) I'll look into how capable vispy is, and how much work we'll need to do to get it to work. FWIW, we already would have had to do some work to get pyqtgraph to have all the functionality we need.

[mwaskom]

Twist!

[mluessi]

..for some reason I dropped off the PySurfer watchlist so I'm late to the party.

I think it's a great idea to try to rewrite parts of PySurfer to use different backends, e.g., vispy or some abstraction layer for xtk, which may make it possible to directly use PySurfer in IPython notebooks (which would be a killer feature). As I see it, the xtk backend would probably generate java-script code on the fly which is then sent to the notebook for execution. Maybe @haehn will be able to help with this.

[mluessi]

PS: And, as always, I blame Apple (and the late Steve Jobs) for mayavi not supporting Py3

http://www.vtk.org/pipermail/vtkusers/2013-April/078934.html

[mwaskom]

"I'm not porting my library until OSs ship with Python 3"

"We're not shipping with Python 3 until it has full library support"

And that's why we're in this mess.

On Tue, Dec 10, 2013 at 8:26 AM, Martin Luessi notifications@github.comwrote:

PS: And, as always, I blame Apple (and the late Steve Jobs) for mayavi not supporting Py3

http://www.vtk.org/pipermail/vtkusers/2013-April/078934.html

— Reply to this email directly or view it on GitHubhttps://github.com/nipy/PySurfer/issues/77#issuecomment-30242463 .

[mluessi]

@mwaskom I was joking regarding apple. The VTK logic is pretty silly, especially given that VTK is cross platform and most people who use OS-X for scientific computing don't use the Python version that ships with OS-X anyways, oh well.

[mwaskom]

No I know I was just remarking on how obvious, at least in retrospect, this problem seems.

[mwaskom]

Just wanted to poke at this issue and say that I've recently been playing around a lot with more "interactive" uses of PySurfer, e.g. https://github.com/mwaskom/connexplore. Will vispy allow this kind of mouse-based interaction?

[larsoner]

I haven't tried that bit of code, but I can say that you can define any type of mapping between mouse movements / clicks / etc. you want and visual operations (resizing, panning, etc.) by defining different camera classes. vispy has a few reasonable defaults built in already that you can choose from, too.

[mwaskom]

Cool, good to know that they'll have some interactivity. I think the ability to click on the brain mesh and get a reference to where you clicked is important to a lot of functionality currently absent in pysurfer (but easy to build on top of).

[larsoner]

Closing this as a dup of #106 (even though that one came later the discussion is more extensive)