jlstevens
commented
10 years ago This is still a valid review of VisTrails (and may be useful for future reference) but we have opted to use IPython Notebook to replace the GUI. There is no actual problem to address so I shall now close this issue.
Converted from SourceForge issue 3537810, submitted by jlstevens Submit Date: 2012-06-25 14:58 GMT
I have investigated the VisTrails package (http://www.vistrails.org/) by testing the beta version (v2.0) and concluded that Topographica integration with VisTrails may offer a way of replacing parts of the Topographica GUI withouthaving to rewrite new GUI components from scratch. VisTrails is written in Python and is under active development with core dependencies on PyQt4, numpy and scipy. Many tutorial examples use VTK (a rather large dependency)but this is not strictly necessary to use the program. Although I only tested the beta version, the software is fairly polished and stable, the documentation is also excellent and installation was very easy.
The core of VisTrails is based on a dataflow paradigm where you create boxes with input and output ports and connect them up appropriately. Most of the examples I saw involves the generation/manipulation and visualisation of 3Ddatasets using VTK. It also has tools for web access so a 'workflow' may involve downloading compressed data from a URL, decompressing and loading the dataset, manipulating it and visualising it. It can also access a relationaldatabase (unfortunately, no HDF5 support that I could see). The essential idea is that user can describe (often elaborate) workflows involving many different tools using node primitives offered by VisTrails 'packages'.There is also excellent functionality for tweaking parameters and excellent tools for easily viewing and comparing results.
VisTrails can achieve fairly complex workflows using the GUI alone and you can write your own PythonSource 'modules' (nodes in the workflow graph)for more control using the inbuilt editor. There is an excellent system for versioningand searching your dataflow graphs and supports persistent storage of all the changes you make as you work. Multiple visual data formats (raster images, pylab plots, SVG graphics, 3D models) can be displayed in the'spreadsheet' view which could potentially be used to display the maps generated by Topographica simulations. This 'spreadsheet' approach is particularly powerful for visualising 3D data sets and you can build 'mashups' thatexpose widget controls allowing the user to easily manipulate the results interactively.
Using SVG and Latex to display the results is probably more practical than using VisTrails for normal Topographica usage. However, a Topographica VistTrails 'package' could expose Topographica components in a new GUI with relativelylittle overhead. For example, a VisTrails node may have the ability to load a Topographica snapshot and connect to a set of analysis nodes ('measure_or_pref', 'measure_od_pref, 'measure_cog' etc) before displaying theresulting maps/figures in the spreadsheet view. This would allow easy side-by-side comparison of different settings and use of VisTrails versioning system. The important question is deciding which TOpographica components would beuseful to use in this way.Is there any common case where you need to explore the effect of the 'num_phase' or 'num_orientation' parameters on the resulting OR map for example? In my own experience, most of the time spent tuningparameters in Topographica is spent tuning the underlying model rather than the measurement protocols.
As tuning Topographica models (especially developmental models) requires a lengthy simulation before the result are made available, it is hard to see how entire Topographica models could be tuned interactively within VisTrails. VisTrails shines when you have lots of compositional operators which need tuning and which don't take long to process. Topographica's pattern classes do fit this description and for this reason I think VisTrails could do a good jobediting, composing and exploring patterns, replacing the existing test pattern editor. Analysis (measurement) functions don't take too long to process but generally they aren't compositional - locating pinwheels/calculatingpinwheel density on an OR map is one exception that I know of.
I can also imagine VisTrails being used to replace the Topographica model editor by providing a set of nodes corresponding to sheets, connections, transfer functions, visualisations etc that are then connected togetherappropriately.Visual output would have to wait till the simulation completes to be displayed but it would still be nice to use the excellent comparison and versioning tools VisTrails has to offer. Such an approach wouldn'treplace .ty model definitions but it could be nicer way of quickly defining models than using the current model editor.
In summary, with a concentrated effort I think VisTrails could serve to replace parts of the Topographica GUI, namely the test pattern editor (especially if this component is factored out as a separate project) and potentially themodel editor. Most of the work would be finding the right dataflow design for Topographica - the VisTrails Python API is fairly simple and most of the effort would involve wrapping existing Topographica functionality (ie decidingthe inputs and outputs appropriately). One issue to consider is that although numpy datatypes can be used with VisTrail modules, this requires a separate package outside the core VisTrails distribution (though available fromthe main VisTrails website). Lastly, I would highly recommend considering useage of VisTrails if we ever need to view or manipulate 3D data.
NOTE: An attempt to submit this feature request was made on 17/05/12 but for some reason did not appear in the tracker. This is a resubmission of that request.