search

mne-tools / mne-python

MNE: Magnetoencephalography (MEG) and Electroencephalography (EEG) in Python
https://mne.tools
BSD 3-Clause "New" or "Revised" License
2.68k stars 1.31k forks source link

Failing example: plot_epochs_as_data_frame.py #286

Closed larsoner closed 11 years ago

larsoner commented 11 years ago

I have a failing example:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python2.7/dist-packages/spyderlib/widgets/externalshell/sitecustomize.py", line 487, in runfile
    execfile(filename, namespace)
  File "/home/larsoner/custombuilds/mne-python/examples/export/plot_epochs_as_data_frame.py", line 176, in <module>
    df.set_index('condition', append=True, inplace=True)
TypeError: set_index() got an unexpected keyword argument 'append'

Thoughts @christianmbrodbeck @dengemann? I am running Ubuntu 12.10 (latest) with pandas from the repos, so I think the version (pandas 0.8.0) should probably be supported...

dengemann commented 11 years ago

... incredible, I was developing on 0.8.1 .... damn it. You are right, going through the versions was something i still wanted to do before 0.5.... Can try to find a fix tomorrow.

larsoner commented 11 years ago

Versions are such a pain. At least across all our computers we can catch a lot of them before subjecting the general public to them too much.

dengemann commented 11 years ago

instead append=True --- drop=False coudl do it.

On Tue, Dec 18, 2012 at 1:38 AM, Denis-Alexander Engemann < denis.engemann@gmail.com> wrote:

... incredible, I was developing on 0.8.1 .... damn it. You are right, going through the versions was something i still wanted to do before 0.5.... Can try to find a fix tomorrow.

dengemann commented 11 years ago

Yes, that's then positive aspect of it ;-)

On Tue, Dec 18, 2012 at 1:40 AM, Eric89GXL notifications@github.com wrote:

Versions are such a pain. At least across all our computers we can catch a lot of them before subjecting the general public to them too much.

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/286#issuecomment-11469120.

larsoner commented 11 years ago

Is drop=False forward-compatible, or do I need to put it in a try-except?

christianbrodbeck commented 11 years ago

On OS X 10.8.2 with EPD 32 / pandas 0.9.1-1 it runs fine. Do you want me to try other versions? Is there no pl.show() at the end intentionally?

dengemann commented 11 years ago

I should install a virtualbox with debian stable or so...

On Tue, Dec 18, 2012 at 1:41 AM, Denis-Alexander Engemann < denis.engemann@gmail.com> wrote:

Yes, that's then positive aspect of it ;-)

On Tue, Dec 18, 2012 at 1:40 AM, Eric89GXL notifications@github.comwrote:

Versions are such a pain. At least across all our computers we can catch a lot of them before subjecting the general public to them too much.

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/286#issuecomment-11469120.

dengemann commented 11 years ago

. on my box pandas would just show it .... but maybe adding it would make sense. Let me tell you in a few moments whether it is forward compatible...

On Tue, Dec 18, 2012 at 1:42 AM, Christian Brodbeck < notifications@github.com> wrote:

On OS X 10.8.2 with EPD 32 / pandas 0.9.1-1 it runs fine. Do you want me to try other versions? Is there no pl.show() at the end intentionally?

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/286#issuecomment-11469176.

larsoner commented 11 years ago

@dengemann drop=False works, but then I get another error:

ValueError: level > 0 only valid with MultiIndex

  File "/home/larsoner/custombuilds/mne-python/examples/export/plot_epochs_as_data_frame.py", line 182, in <module>
    grouped = df.groupby(level=['condition', 'epoch'])
  File "/usr/lib/pymodules/python2.7/pandas/core/generic.py", line 134, in groupby
    sort=sort, group_keys=group_keys)
  File "/usr/lib/pymodules/python2.7/pandas/core/groupby.py", line 459, in groupby
    return klass(obj, by, **kwds)
  File "/usr/lib/pymodules/python2.7/pandas/core/groupby.py", line 142, in __init__
    level=level, sort=sort)
  File "/usr/lib/pymodules/python2.7/pandas/core/groupby.py", line 1055, in _get_grouper
    raise ValueError('level > 0 only valid with MultiIndex')
christianbrodbeck commented 11 years ago

. on my box pandas would just show it .... but maybe adding it would make

here the plot just blinks and disappears (unlike other examples)

dengemann commented 11 years ago

forget about drop=False...

On Tue, Dec 18, 2012 at 1:43 AM, Denis-Alexander Engemann < denis.engemann@gmail.com> wrote:

. on my box pandas would just show it .... but maybe adding it would make sense. Let me tell you in a few moments whether it is forward compatible...

On Tue, Dec 18, 2012 at 1:42 AM, Christian Brodbeck < notifications@github.com> wrote:

On OS X 10.8.2 with EPD 32 / pandas 0.9.1-1 it runs fine. Do you want me to try other versions? Is there no pl.show() at the end intentionally?

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/286#issuecomment-11469176.

dengemann commented 11 years ago

I fear i've got to modify the example a bit. It's just about one or two lines but the rest seems to depend on it and so the story told somewhat changes.

On Tue, Dec 18, 2012 at 1:45 AM, Eric89GXL notifications@github.com wrote:

@dengemann https://github.com/dengemann drop=False works, but then I get another error:

ValueError: level > 0 only valid with MultiIndex

File "/home/larsoner/custombuilds/mne-python/examples/export/plot_epochs_as_data_frame.py", line 182, in grouped = df.groupby(level=['condition', 'epoch']) File "/usr/lib/pymodules/python2.7/pandas/core/generic.py", line 134, in groupby sort=sort, group_keys=group_keys) File "/usr/lib/pymodules/python2.7/pandas/core/groupby.py", line 459, in groupby return klass(obj, by, kwds) File "/usr/lib/pymodules/python2.7/pandas/core/groupby.py", line 142, in init** level=level, sort=sort) File "/usr/lib/pymodules/python2.7/pandas/core/groupby.py", line 1055, in _get_grouper raise ValueError('level > 0 only valid with MultiIndex')

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/286#issuecomment-11469245.

dengemann commented 11 years ago

How does this work on your boxes?

https://gist.github.com/b12c637da892511b0862

On Tue, Dec 18, 2012 at 1:49 AM, Denis-Alexander Engemann < denis.engemann@gmail.com> wrote:

I fear i've got to modify the example a bit. It's just about one or two lines but the rest seems to depend on it and so the story told somewhat changes.

On Tue, Dec 18, 2012 at 1:45 AM, Eric89GXL notifications@github.comwrote:

@dengemann https://github.com/dengemann drop=False works, but then I get another error:

ValueError: level > 0 only valid with MultiIndex

File "/home/larsoner/custombuilds/mne-python/examples/export/plot_epochs_as_data_frame.py", line 182, in grouped = df.groupby(level=['condition', 'epoch']) File "/usr/lib/pymodules/python2.7/pandas/core/generic.py", line 134, in groupby sort=sort, group_keys=group_keys) File "/usr/lib/pymodules/python2.7/pandas/core/groupby.py", line 459, in groupby return klass(obj, by, kwds) File "/usr/lib/pymodules/python2.7/pandas/core/groupby.py", line 142, in init** level=level, sort=sort) File "/usr/lib/pymodules/python2.7/pandas/core/groupby.py", line 1055, in _get_grouper raise ValueError('level > 0 only valid with MultiIndex')

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/286#issuecomment-11469245.

dengemann commented 11 years ago

changed: df = df.reset_index('epoch')

and

max_latency = grouped[sel[0]].apply(lambda x: x.index[x.argmax()])

-- if it's fine let me clean it up this way tomorrow.

On Tue, Dec 18, 2012 at 1:54 AM, Denis-Alexander Engemann < denis.engemann@gmail.com> wrote:

How does this work on your boxes?

https://gist.github.com/b12c637da892511b0862

On Tue, Dec 18, 2012 at 1:49 AM, Denis-Alexander Engemann < denis.engemann@gmail.com> wrote:

I fear i've got to modify the example a bit. It's just about one or two lines but the rest seems to depend on it and so the story told somewhat changes.

On Tue, Dec 18, 2012 at 1:45 AM, Eric89GXL notifications@github.comwrote:

@dengemann https://github.com/dengemann drop=False works, but then I get another error:

ValueError: level > 0 only valid with MultiIndex

File "/home/larsoner/custombuilds/mne-python/examples/export/plot_epochs_as_data_frame.py", line 182, in grouped = df.groupby(level=['condition', 'epoch']) File "/usr/lib/pymodules/python2.7/pandas/core/generic.py", line 134, in groupby sort=sort, group_keys=group_keys) File "/usr/lib/pymodules/python2.7/pandas/core/groupby.py", line 459, in groupby return klass(obj, by, kwds) File "/usr/lib/pymodules/python2.7/pandas/core/groupby.py", line 142, in init** level=level, sort=sort) File "/usr/lib/pymodules/python2.7/pandas/core/groupby.py", line 1055, in _get_grouper raise ValueError('level > 0 only valid with MultiIndex')

— Reply to this email directly or view it on GitHubhttps://github.com/mne-tools/mne-python/issues/286#issuecomment-11469245.

christianbrodbeck commented 11 years ago

Works as before, still not keeping the figure. Even adding pl.show() at the end does not actually make the figure stay.

larsoner commented 11 years ago 
    execfile(filename, namespace)
  File "/home/larsoner/custombuilds/mne-python/examples/export/plot_epochs_as_data_frame.py", line 182, in <module>
    grouped = df.groupby(['condition', 'epoch'])
  File "/usr/lib/pymodules/python2.7/pandas/core/generic.py", line 134, in groupby
    sort=sort, group_keys=group_keys)
  File "/usr/lib/pymodules/python2.7/pandas/core/groupby.py", line 459, in groupby
    return klass(obj, by, **kwds)
  File "/usr/lib/pymodules/python2.7/pandas/core/groupby.py", line 142, in __init__
    level=level, sort=sort)
  File "/usr/lib/pymodules/python2.7/pandas/core/groupby.py", line 1110, in _get_grouper
    gpr = obj[gpr]
  File "/usr/lib/pymodules/python2.7/pandas/core/frame.py", line 1634, in __getitem__
    return self._get_item_cache(key)
  File "/usr/lib/pymodules/python2.7/pandas/core/generic.py", line 473, in _get_item_cache
    values = self._data.get(item)
  File "/usr/lib/pymodules/python2.7/pandas/core/internals.py", line 827, in get
    _, block = self._find_block(item)
  File "/usr/lib/pymodules/python2.7/pandas/core/internals.py", line 935, in _find_block
    self._check_have(item)
  File "/usr/lib/pymodules/python2.7/pandas/core/internals.py", line 942, in _check_have
    raise KeyError('no item named %s' % str(item))
KeyError: 'no item named epoch'
larsoner commented 11 years ago

Line 182, the failing one, is this:

grouped = df.groupby(['condition', 'epoch'])
dengemann commented 11 years ago

pandas is definitely developping fast.... gr. let me fix it tomorrow in trying to bypass these multi index related opereations. the feature is apparently too unsettled to be good example material. at least i'm happy the exporter itself worked on your boxes.

anyways, should not be difficult to fix. has anyone 7 < pandas < 8 on his box?

On 18.12.2012, at 01:58, Eric89GXL notifications@github.com wrote:

Line 182, the failing one, is this:

grouped = df.groupby(['condition', 'epoch']) — Reply to this email directly or view it on GitHub.

larsoner commented 11 years ago

I only have 0.8.0. It looks like pandas' github has maintenance branches for 0.7.x and 0.9.x, though, so presumably if you develop for 0.7.x you'll be okay.

christianbrodbeck commented 11 years ago

EPD has

pandas             0.2-2              free                 
                   0.2-3              free                 
                   0.3.0-1            free                 
                   0.3.0-2            free                 
                   0.3.0-3            free                 
                   0.4.3-1            free                 
                   0.6.1-1            free                 
                   0.7.0-1            free                 
                   0.7.1-1            free                 
                   0.7.2-1            free                 
                   0.7.3-1            free                 
                   0.7.3-2            free                 
                   0.8.0rc2-1         free                 
                   0.9.0-1            free                 
                 * 0.9.1-1            free
dengemann commented 11 years ago

yes, 0.7 would also be my intuition about the lower bound for support. i think changes since 0.5 / 0.6 have been rather dramatic. Thanks for trying / reporting! I'll test against the supported branches hoping to find a common denominator. as i said i'm sure the multi-index api is causing the most trouble.

thx, d

On 18.12.2012, at 02:14, Eric89GXL notifications@github.com wrote:

I only have 0.8.0. It looks like pandas' github has maintenance branches for 0.7.x and 0.9.x, though, so presumably if you develop for 0.7.x you'll be okay.

— Reply to this email directly or view it on GitHub.

christianbrodbeck commented 11 years ago

Running your pandas_example.py with 0.7.2-1 I get the following, 

Traceback (most recent call last):
  File "pandas_example.py", line 159, in <module>
    df.ix[:3, sel].plot(xticks=xticks)
  File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/pandas/core/frame.py", line 3923, in plot
    ax.plot(x, y, label=str(col), **kwds)
  File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py", line 3848, in plot
    for line in self._get_lines(*args, **kwargs):
  File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py", line 323, in _grab_next_args
    for seg in self._plot_args(remaining, kwargs):
  File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py", line 310, in _plot_args
    seg = func(x[:,j%ncx], y[:,j%ncy], kw, kwargs)
  File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py", line 260, in _makeline
    self.set_lineprops(seg, **kwargs)
  File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py", line 201, in set_lineprops
    raise TypeError, 'There is no line property "%s"'%key
TypeError: There is no line property "xticks"
dengemann commented 11 years ago

thanks, that's very useful. so fancy plotting args are the second thing to target at....

On 18.12.2012, at 02:20, Christian Brodbeck notifications@github.com wrote:

Running your pandas_example.py with 0.7.2-1 I get the following,

Traceback (most recent call last): File "pandas_example.py", line 159, in df.ix[:3, sel].plot(xticks=xticks) File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/pandas/core/frame.py", line 3923, in plot ax.plot(x, y, label=str(col), _kwds) File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py", line 3848, in plot for line in self._get_lines(_args, _kwargs): File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py", line 323, in _grab_next_args for seg in self._plot_args(remaining, kwargs): File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py", line 310, in _plot_args seg = func(x[:,j%ncx], y[:,j%ncy], kw, kwargs) File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py", line 260, in _makeline self.set_lineprops(seg, *_kwargs) File "/Library/Frameworks/Python.framework/Versions/7.3/lib/python2.7/site-packages/matplotlib/axes.py", line 201, in set_lineprops raise TypeError, 'There is no line property "%s"'%key TypeError: There is no line property "xticks" — Reply to this email directly or view it on GitHub.

dengemann commented 11 years ago

Hi folks,

before cleaning up the docs and issuing a PR, if you can spare 5 minutes, please tell me whether this works on your boxes:

I tested it against pandas 0.9.1 and 0.7.3

https://gist.github.com/b12c637da892511b0862

runs smooth here. If all is fine, my cut off suggestion would be pandas 0.7.3 which is about one year old. I think people knowing pandas also know how experimental / vibrant this project is, and probably are frequent updaters themselves.

Wdyt?

larsoner commented 11 years ago

That sounds like a reasonable cutoff to me.

Ran fine, popped up 3 plots. You want to see them?

christianbrodbeck commented 11 years ago

With 0.7.2-1 I still get the same error, with 0.7.3-1 it runs without error (although the plots still disappear instantly)

larsoner commented 11 years ago

@christianmbrodbeck do the plots in other examples work for you? The plots stick around for me using Matplotlib 1.1.1 (under Spyder 2.1.10).

christianbrodbeck commented 11 years ago

yes the plots of other examples stay open (and block the interpreter) until I close them. Just confirmed this with plot_topography.py, plot_topo_compare_conditions.py, plot_simulate_evoked_data.py (anyway in the pyshell that I usually use everything works fine, they only close quickly if run from the terminal)

christianbrodbeck commented 11 years ago

current mpl in my EPD is 1.1.0-2

dengemann commented 11 years ago

grr, should be 4...

On 19.12.2012, at 00:02, Eric89GXL notifications@github.com wrote:

That sounds like a reasonable cutoff to me.

Ran fine, popped up 3 plots. You want to see them?

— Reply to this email directly or view it on GitHub.

christianbrodbeck commented 11 years ago

I get 4 in pyshell :)

dengemann commented 11 years ago

... too mysterious. i added pl.ion() and ran it across versions and operating sytems (debian, mac)...

would be good to know what goes on on your box, chris.

what if you just ad pl.show after one of the plotting command?

hope we will identify the source of the trouble...

anyways, thanks for your responses!

D

On 19.12.2012, at 00:24, Christian Brodbeck notifications@github.com wrote:

yes the plots of other examples stay open (and block the interpreter) until I close them. Just confirmed this with plot_topography.py, plot_topo_compare_conditions.py, plot_simulate_evoked_data.py (anyway in the pyshell that I usually use everything works fine, they only close quickly if run from the terminal)

— Reply to this email directly or view it on GitHub.

mluessi commented 11 years ago

"With 0.7.2-1 I still get the same error, with 0.7.3-1 it runs without error" same here.

dengemann commented 11 years ago

aha, interesting. diversities of python.... but still, no real pattern. is it a bar plot or an meg plot that additionally appeated now?

On 19.12.2012, at 00:26, Christian Brodbeck notifications@github.com wrote:

I get 4 in pyshell :)

— Reply to this email directly or view it on GitHub.

larsoner commented 11 years ago

Maybe it was 4, I will double-check

larsoner commented 11 years ago

It was four! My bad. Two bar plots (one vert one horiz), plus two plots with MEG sensors. Just noticed there aren't units on most of the axes (ms for RT presumably, and fT/cm (?) for MEG), so you should check that, too.

christianbrodbeck commented 11 years ago

I now put a pl.show() behind every tight-layout call, tried different backends (macosx, TkAgg, WXAgg), and EPD 64-bit, but nothing makes the plots stay when run from the terminal --- strange

dengemann commented 11 years ago

no, there aren't. did not want to overdo it, keep it somewhat raw and leave units + polish to our native routines. do you want me to add units?

On 19.12.2012, at 00:35, Eric89GXL notifications@github.com wrote:

It was four! My bad. Two bar plots (one vert one horiz), plus two plots with MEG sensors. Just noticed there aren't units on most of the axes (ms for RT presumably, and fT/cm (?) for MEG), so you should check that, too.

— Reply to this email directly or view it on GitHub.

larsoner commented 11 years ago

I'm fine with no units if you think it adds unnecessary complexity.

dengemann commented 11 years ago

that's really strange... is this on a mac? did you make sure not to run the example in fullscreen mode terminal?

On 19.12.2012, at 00:52, Christian Brodbeck notifications@github.com wrote:

I now put a pl.show() behind every tight-layout call, tried different backends (macosx, TkAgg, WXAgg), and EPD 64-bit, but nothing makes the plots stay when run from the terminal --- strange

— Reply to this email directly or view it on GitHub.

larsoner commented 11 years ago

If you think adding them would add unnecessary complexity, that is.

christianbrodbeck commented 11 years ago

that's really strange... is this on a mac? did you make sure not to run the example in fullscreen mode terminal?

10.8.2. Fullscreen mode terminal (you mean the native Lion full screen mode?) does not help either.

dengemann commented 11 years ago

we could add it, no extra complexity. my intention was just to put the focus on quick one liners and expose pandas as interactive tool rather then as plotting routine; i tried to add as few lines as possible to convey the pandas feel. if you think the examples would benifit and gain beauty from more detail i'm happy to add units ;-)

On 19.12.2012, at 01:10, Eric89GXL notifications@github.com wrote:

If you think adding them would add unnecessary complexity, that is.

— Reply to this email directly or view it on GitHub.

larsoner commented 11 years ago

I'll let the merge-master @agramfort decide :)

dengemann commented 11 years ago

... the other way round, the os x fullscreen would hide my plots....

On 19.12.2012, at 01:12, Christian Brodbeck notifications@github.com wrote:

that's really strange... is this on a mac? did you make sure not to run the example in fullscreen mode terminal?

10.8.2. Fullscreen mode terminal (you mean the native Lion full screen mode?) does not help either.

— Reply to this email directly or view it on GitHub.

christianbrodbeck commented 11 years ago

... the other way round, the os x fullscreen would hide my plots....

oh, no, it actually took me back to the desktop when the plots appeared :) but they did not stay

christianbrodbeck commented 11 years ago

somebody else has mac/EPD to confirm this? If it's just something about my specific configuration maybe it's not worth the trouble

dengemann commented 11 years ago

Anyways, although preferring a raw style here, i'm not to passionate about this. As soon as some one says he wants units i'll just add them ;-)

On 19.12.2012, at 01:19, Eric89GXL notifications@github.com wrote:

I'll let the merge-master @agramfort decide :)

— Reply to this email directly or view it on GitHub.

dengemann commented 11 years ago

i run mac epd on 10.8 ...

what happens id you just open a fresh terminal, construct a cartoon df from a random 2d array and invoke the plot method?

On 19.12.2012, at 01:21, Christian Brodbeck notifications@github.com wrote:

somebody else has mac/EPD to confirm this? If it's just something about my specific configuration maybe it's not worth the trouble

— Reply to this email directly or view it on GitHub.

christianbrodbeck commented 11 years ago 
[christian@Christians-MacBook-Pro] ~/Documents/Eclipse/projects/mne-python/examples$ python
Enthought Python Distribution -- www.enthought.com
Version: 7.3-2 (32-bit)

Python 2.7.3 |EPD 7.3-2 (32-bit)| (default, Apr 12 2012, 11:28:34) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "credits", "demo" or "enthought" for more information.
>>> from pylab import *
>>> import numpy as np
>>> plot(np.random.normal(0,1,5))
[<matplotlib.lines.Line2D object at 0x5b406f0>]

here the rocket app icon appears but I'm still in the terminal

>>> show()

Now the plot shows up until I dismiss it