pyUpSet

A pure-python implementation of the UpSet suite of visualisation methods by Lex, Gehlenborg et al.

Purpose

The purpose of this package is to statically reproduce some of the visualisations that can be obtained through the UpSet tool of Lex, Gehlenborg et al.

In particular, pyUpSet strengthens UpSet's focus on intersections, which motivates many of the design choices behind the exposed interface and the internal mechanics of the module. (More on this below.)

Consistently with the documentation used for Lex et al.'s UpSet, the data employed in the following examples comes from the movie data set of the GroupLens Labs.

How to install

pyUpSet is on PyPI and can therefore be installed via pip:

pip install pyupset

If you'd rather install from source, you can download and unzip the tar archive (in pyupset/dist/) and run

python setup.py install

How it works

The current interface is very simple: Plots can be generated solely from the exposed function plot, whose arguments allow flexible customisations of the graphs. The easiest example is the plain, straightforward basic intersection plot:

import pyupset as pyu
from pickle import load
with open('./test_data_dict.pckl', 'rb') as f:
   data_dict = load(f)
pyu.plot(data_dict)

to produce basic plot

N.B.: Notice that intersections are exclusive, meaning that they form a partition of the union of the base sets.

Displayed intersections can also be filtered or sorted by size or degree:

pyu.plot(data_dict, unique_keys = ['title'], sort_by='degree', inters_size_bounds=(20, 400))

produces basic filtering

The example above also uses the unique_keys kwarg, which specifies columns of the underlying data frames in data_dict that can be used to uniquely identify rows and possibly speed up the computation of intersections.

Intersection highlighting

pyUpSet supports "queries", i.e. the highlighting of intersections. Intersections to highlight are specified through tuples. For example, the following call produces graphs where all data is highlighted that corresponds to movies classified as both "adventure" and "action", or "romance" and "war".

pyu.plot(data_dict, unique_keys = ['title'], 
         additional_plots=[{'kind':'scatter', 'data_quantities':{'x':'views', 'y':'rating_std'}},
                           {'kind':'hist', 'data_quantities':{'x':'views'}}],
         query = [('adventure', 'action'), ('romance', 'war')]
        )

simple query

Additional plots

It is possible to add further plots that use information contained in the data frames, as in

pyu.plot(data_dict, unique_keys = ['title'], 
         additional_plots=[{'kind':'scatter', 'data_quantities':{'x':'views', 'y':'rating_std'}},
                           {'kind':'hist', 'data_quantities':{'x':'views'}}]), 
         query = [('adventure', 'action'), ('romance', 'war')]

This produces additional plots with query

The highlighting produced by the queries is passed to the additional graphs. The dictionary specifying the additional graphs can also take standard matplotlib arguments as kwargs:

 pyu.plot(data_dict, unique_keys = ['title'], 
         additional_plots=[{'kind':'scatter', 
                            'data_quantities':{'x':'views', 'y':'rating_std'},
                            'graph_properties':{'alpha':.8, 'lw':.4, 'edgecolor':'w', 's':50}},
                           {'kind':'hist', 
                            'data_quantities':{'x':'views'},
                            'graph_properties':{'bins':50}}], 
         query = [('adventure', 'action'), ('romance', 'war')])

yields additional plots with query and properties

A note on the input format

pyUpSet has a very specific use case: It is focussed on the study of intersections of sets. In order for a definition of intersection to make sense, and even more for the integration of additional graphs to be meaningful, it is assumed that the input data frames have properties of homonymy (they contain columns with the same names) and homogeneity (columns with the same name, intuitively, contain data of the same kind). While hononymy is a purely interface-dependent requirement whose aim is primarily to make pyUpSet's interface leaner, homogeneity has a functional role in allowing definitions of uniqueness and commonality for the data points in the input data frames.

Whenever possible, pyUpSet will try to check for (and enforce) the two above properties. In particular, when the unique_keys argument of plot is omitted, pyUpSet will try to use all columns with common names across the data frames as a list of unique keys. Under the hypotheses of homogeneity and homonymy this should be enough for all the operations carried out by pyUpSet to complete successfully.

Upcoming changes

Please bear in mind that pyUpset is under active development so current behaviour may change at any time. In particular, here is a list of changes, in no particular order, to be expected soon:

improved OO interface for increased flexibility and customisation
improved, automated scaling of figure and axes grid according to the number of sets, intersections and additional plots (at the moment manual resizing may be needed)

ImSoErgodic / py-upset

readme