Plotting ica properties returns ValueError (dtype 64)

[nrosjat]

I'm trying to follow the Artifact Correction with ICA example with Python 2.7.14 under Spyder in 64-bit mode (conda version 4.3.29). Everything works as expected until:

ica.plot_properties(raw, picks=0)

which leads to an ValueError.

File "", line 1, in ica.plot_properties(raw,picks=picks)

File "...\Anaconda3\envs\py27\lib\site-packages\mne\preprocessing\ica.py", line 1435, in plot_properties figsize=figsize, show=show)

File "...\Anaconda3\envs\py27\lib\site-packages\mne\viz\ica.py", line 241, in plot_ica_properties psds, freqs = psd_multitaper(epochs_src, picks=picks, **psd_args)

File "", line 2, in psd_multitaper

File "...\Anaconda3\envs\py27\lib\site-packages\mne\utils.py", line 728, in verbose return function(*args, **kwargs)

File "...\Anaconda3\envs\py27\lib\site-packages\mne\time_frequency\psd.py", line 292, in psd_multitaper n_jobs=n_jobs, verbose=verbose)

File "", line 2, in psd_array_multitaper

File "...\Anaconda3\envs\py27\lib\site-packages\mne\utils.py", line 728, in verbose return function(*args, **kwargs)

File "...\Anaconda3\envs\py27\lib\site-packages\mne\time_frequency\multitaper.py", line 521, in psd_array_multitaper low_bias=low_bias)

File "...\Anaconda3\envs\py27\lib\site-packages\mne\time_frequency\multitaper.py", line 201, in dpss_windows select_range=(N - Kmax, N - 1))

File "...\Anaconda3\envs\py27\lib\site-packages\scipy\linalg\decomp.py", line 847, in eigvals_banded return eig_banded(a_band, lower=lower, eigvals_only=1,

File "...\Anaconda3\envs\py27\lib\site-packages\scipy\linalg\decomp.py", line 587, in eig_banded raise ValueError('expected two-dimensional array')

File "...\Anaconda3\envs\py27\lib\site-packages\scipy\linalg\decomp.py", line 478, in _check_select if sr.dtype.char.lower() not in 'lih':

ValueError: when using select="i", select_range must contain integers, got dtype int64

I'm quite new to Python, but I guess it has something to do with using the 64-bit version? I tried it with 32-bit mode, but then I ended up with a MemoryError. Does anyone have a suggestion on how to resolve this problem?

[larsoner]

This one is a bit weird. Are you on Windows?

Can you change line 509 of mne/time_frequency/multitaper.py from:

n_times = x.shape[-1]

to

n_times = int(x.shape[-1])

To see if it fixes the problem?

[nrosjat]

Are you on Windows?

Yes I am using Windows 7 (64-Bit).

To see if it fixes the problem?

I changed the line accordingly, but the error still remains the same.

[nrosjat]

If I skip this step and move on to the artifact detection using EOG information multitaper.py produces another error:

File "...\py27\lib\site-packages\mne\time_frequency\multitaper.py", line 509, in psd_array_multitaper dshape = int(x.shape[:-1]) TypeError: int() argument must be a string or a number, not 'tuple'

[agramfort]

windows does not use standard int64. One fix is certain to do a np.astype somewhere but for this you need to debug after putting a breakpoint at the right place.

what is the value of sr.dtype ?

[nrosjat]

what is the value of sr.dtype ?

The value of sr.dtype is int64

Somehow the tuple select_range from multitaper.py (line 201) ends up as a dstype = int64 via np.asarray(select_range)?

[agramfort]

no idea

[nrosjat]

I checked for sr.dtype.char.lower() and it is set to 'q' which is not in 'lih'. When adding q to the string the correct figure would be produced.

If I just create the numpyarray manually and check for dtype.char I get the following:

sr.astype(np.int64) Out[21]: array([1], dtype=int64)

sr.dtype.char.lower() Out[22]: 'l'

I have no idea why the type is 'q' in one case, but 'l' in the other.

[larsoner]

Me neither, but let's go ahead and add it to the list (it's c long long equiv)

https://docs.scipy.org/doc/numpy-1.13.0/reference/arrays.scalars.html

[larsoner]

... oh wait it's in a SciPy function. You need to cast it in MNE to make it happy, maybe pass np.int64(N) or something?

[larsoner]

BTW doing sr.astype(np.int64) returns something, it does not change sr in place

[nrosjat]

After adding np.int64(N) to line 201 in multitaper.py the value error is solved:

w = linalg.eigvals_banded(ab, select='i', select_range=(np.int64(N) - Kmax, np.int64(N) - 1))

[davide-aloi]

How can I fix this issue? I am struggling with the same problem!

[agramfort]

how do we replicate? can you replicate on sample data?

[davide-aloi]

Yes, both sample data using this script https://martinos.org/mne/stable/auto_examples/preprocessing/plot_run_ica.html#sphx-glr-auto-examples-preprocessing-plot-run-ica-py and on my data give me the same issue mentioned before. I changed those lines suggested by nrosjat in the multitaper.py file, but I keep getting the same value error.

[davide-aloi]

`--------------------------------------------------------------------------- ValueError Traceback (most recent call last)

in () 56 ######################################################################## ####### 57 # Plot properties of ECG components: ---> 58 ica.plot_properties(epochs, picks=ecg_inds) D:\Users\Davide\Anaconda2\lib\site-packages\mne\preprocessing\ica.pyc in plot_pr operties(self, inst, picks, axes, dB, plot_std, topomap_args, image_args, psd_ar gs, figsize, show) 1433 topomap_args=topomap_args, 1434 image_args=image_args, psd_args=psd_a rgs, -> 1435 figsize=figsize, show=show) 1436 1437 @copy_function_doc_to_method_doc(plot_ica_sources) D:\Users\Davide\Anaconda2\lib\site-packages\mne\viz\ica.pyc in plot_ica_properti es(ica, inst, picks, axes, dB, plot_std, topomap_args, image_args, psd_args, fig size, show) 239 plot_lowpass_edge = inst.info['lowpass'] < Nyquist and ( 240 psd_args['fmax'] > inst.info['lowpass']) --> 241 psds, freqs = psd_multitaper(epochs_src, picks=picks, **psd_args) 242 243 def set_title_and_labels(ax, title, xlab, ylab): D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\psd.pyc in psd_mu ltitaper(inst, fmin, fmax, tmin, tmax, bandwidth, adaptive, low_bias, normalizat ion, picks, proj, n_jobs, verbose) D:\Users\Davide\Anaconda2\lib\site-packages\mne\utils.pyc in verbose(function, * args, **kwargs) 726 with use_log_level(verbose_level): 727 return function(*args, **kwargs) --> 728 return function(*args, **kwargs) 729 730 D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\psd.pyc in psd_mu ltitaper(inst, fmin, fmax, tmin, tmax, bandwidth, adaptive, low_bias, normalizat ion, picks, proj, n_jobs, verbose) 290 bandwidth=bandwidth, adaptive=adaptive, 291 low_bias=low_bias, normalization=normali zation, --> 292 n_jobs=n_jobs, verbose=verbose) D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\multitaper.pyc in psd_array_multitaper(x, sfreq, fmin, fmax, bandwidth, adaptive, low_bias, norma lization, n_jobs, verbose) D:\Users\Davide\Anaconda2\lib\site-packages\mne\utils.pyc in verbose(function, * args, **kwargs) 726 with use_log_level(verbose_level): 727 return function(*args, **kwargs) --> 728 return function(*args, **kwargs) 729 730 D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\multitaper.pyc in psd_array_multitaper(x, sfreq, fmin, fmax, bandwidth, adaptive, low_bias, norma lization, n_jobs, verbose) 519 n_tapers_max = int(2 * half_nbw) 520 dpss, eigvals = dpss_windows(n_times, half_nbw, n_tapers_max, --> 521 low_bias=low_bias) 522 523 # decide which frequencies to keep D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\multitaper.pyc in dpss_windows(N, half_nbw, Kmax, low_bias, interp_from, interp_kind) 199 # only calculate the highest Kmax eigenvalues 200 w = linalg.eigvals_banded(ab, select='i', --> 201 select_range=(np.int64(N) - Kmax, np.i nt64(N) - 1)) 202 w = w[::-1] 203 D:\Users\Davide\Anaconda2\lib\site-packages\scipy\linalg\decomp.pyc in eigvals_b anded(a_band, lower, overwrite_a_band, select, select_range, check_finite) 845 return eig_banded(a_band, lower=lower, eigvals_only=1, 846 overwrite_a_band=overwrite_a_band, select=select, --> 847 select_range=select_range, check_finite=check_fini te) 848 849 D:\Users\Davide\Anaconda2\lib\site-packages\scipy\linalg\decomp.pyc in eig_bande d(a_band, lower, eigvals_only, overwrite_a_band, select, select_range, max_ev, c heck_finite) 585 raise ValueError('expected two-dimensional array') 586 select, vl, vu, il, iu, max_ev = _check_select( --> 587 select, select_range, max_ev, a1.shape[1]) 588 del select_range 589 if select == 0: D:\Users\Davide\Anaconda2\lib\site-packages\scipy\linalg\decomp.pyc in _check_se lect(select, select_range, max_ev, max_len) 476 if sr.dtype.char.lower() not in 'lih': 477 raise ValueError('when using select="i", select_range mu st ' --> 478 'contain integers, got dtype %s' % sr.d type) 479 # translate Python (0 ... N-1) into Fortran (1 ... N) with + 1 480 il, iu = sr + 1 ValueError: when using select="i", select_range must contain integers, got dtype int64`

[davide-aloi]

This is the error I get when trying to plot ica properties. Do I have another way to reach that data?

[davide-aloi]

I get the same error when trying to run https://www.martinos.org/mne/stable/auto_examples/connectivity/plot_sensor_connectivity.html#sphx-glr-auto-examples-connectivity-plot-sensor-connectivity-py

[agramfort]

what does:

mne.sys_info()

return on your system? what OS are you using? win32 or win64?

[davide-aloi]

In [2]: mne.sys_info() D:\Users\Davide\Anaconda2\lib\site-packages\h5py__init__.py:36: FutureWarning:

Conversion of the second argument of issubdtype from float to np.floating is deprecated. In future, it will be treated as np.float64 == np.dtype(float).typ e.

Platform: Windows-7-6.1.7601-SP1 Python: 2.7.14 |Anaconda, Inc.| (default, Nov 8 2017, 13:40:45) [MSC v.1 500 64 bit (AMD64)] Executable: D:\Users\Davide\Anaconda2\python.exe CPU: Intel64 Family 6 Model 42 Stepping 7, GenuineIntel: 4 cores Memory: 7.9 GB

mne: 0.15.2 numpy: 1.14.0 {lapack=mkl_rt, blas=mkl_rt} scipy: 1.0.0 matplotlib: 2.1.2

sklearn: 0.19.1 nibabel: 2.2.1 mayavi: 4.5.0 pycuda: Not found skcuda: Not found pandas: 0.22.0

[larsoner]

@nrosjat @Davi93 can you check to see if #5039 fixes your problem?

[davide-aloi]

I applied the changings but I still get this error: 0 bad epochs dropped Loading data for 319 events and 106 original time points ...

ValueError Traceback (most recent call last)

in () 56 ######################################################################## ####### 57 # Plot properties of ECG components: ---> 58 ica.plot_properties(epochs, picks=ecg_inds) D:\Users\Davide\Anaconda2\lib\site-packages\mne\preprocessing\ica.pyc in plot_pr operties(self, inst, picks, axes, dB, plot_std, topomap_args, image_args, psd_ar gs, figsize, show) 1417 topomap_args=topomap_args, 1418 image_args=image_args, psd_args=psd_a rgs, -> 1419 figsize=figsize, show=show) 1420 1421 @copy_function_doc_to_method_doc(plot_ica_sources) D:\Users\Davide\Anaconda2\lib\site-packages\mne\viz\ica.pyc in plot_ica_properti es(ica, inst, picks, axes, dB, plot_std, topomap_args, image_args, psd_args, fig size, show) 239 plot_lowpass_edge = inst.info['lowpass'] < Nyquist and ( 240 psd_args['fmax'] > inst.info['lowpass']) --> 241 psds, freqs = psd_multitaper(epochs_src, picks=picks, **psd_args) 242 243 def set_title_and_labels(ax, title, xlab, ylab): D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\psd.pyc in psd_mu ltitaper(inst, fmin, fmax, tmin, tmax, bandwidth, adaptive, low_bias, normalizat ion, picks, proj, n_jobs, verbose) D:\Users\Davide\Anaconda2\lib\site-packages\mne\utils.pyc in verbose(function, * args, **kwargs) 726 with use_log_level(verbose_level): 727 return function(*args, **kwargs) --> 728 return function(*args, **kwargs) 729 730 D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\psd.pyc in psd_mu ltitaper(inst, fmin, fmax, tmin, tmax, bandwidth, adaptive, low_bias, normalizat ion, picks, proj, n_jobs, verbose) 290 bandwidth=bandwidth, adaptive=adaptive, 291 low_bias=low_bias, normalization=normali zation, --> 292 n_jobs=n_jobs, verbose=verbose) D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\multitaper.py in psd_array_multitaper(x, sfreq, fmin, fmax, bandwidth, adaptive, low_bias, normal ization, n_jobs, verbose) D:\Users\Davide\Anaconda2\lib\site-packages\mne\utils.pyc in verbose(function, * args, **kwargs) 726 with use_log_level(verbose_level): 727 return function(*args, **kwargs) --> 728 return function(*args, **kwargs) 729 730 D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\multitaper.py in psd_array_multitaper(x, sfreq, fmin, fmax, bandwidth, adaptive, low_bias, normal ization, n_jobs, verbose) 493 n_tapers_max = int(2 * half_nbw) 494 dpss, eigvals = dpss_windows(n_times, half_nbw, n_tapers_max, --> 495 low_bias=low_bias) 496 497 # decide which frequencies to keep D:\Users\Davide\Anaconda2\lib\site-packages\mne\time_frequency\multitaper.py in dpss_windows(N, half_nbw, Kmax, low_bias, interp_from, interp_kind) 187 # only calculate the highest Kmax eigenvalues 188 w = linalg.eigvals_banded(ab, select='i', --> 189 select_range=(N - Kmax, N - 1)) 190 w = w[::-1] 191 D:\Users\Davide\Anaconda2\lib\site-packages\scipy\linalg\decomp.pyc in eigvals_b anded(a_band, lower, overwrite_a_band, select, select_range, check_finite) 845 return eig_banded(a_band, lower=lower, eigvals_only=1, 846 overwrite_a_band=overwrite_a_band, select=select, --> 847 select_range=select_range, check_finite=check_fini te) 848 849 D:\Users\Davide\Anaconda2\lib\site-packages\scipy\linalg\decomp.pyc in eig_bande d(a_band, lower, eigvals_only, overwrite_a_band, select, select_range, max_ev, c heck_finite) 585 raise ValueError('expected two-dimensional array') 586 select, vl, vu, il, iu, max_ev = _check_select( --> 587 select, select_range, max_ev, a1.shape[1]) 588 del select_range 589 if select == 0: D:\Users\Davide\Anaconda2\lib\site-packages\scipy\linalg\decomp.pyc in _check_se lect(select, select_range, max_ev, max_len) 476 if sr.dtype.char.lower() not in 'lih': 477 raise ValueError('when using select="i", select_range mu st ' --> 478 'contain integers, got dtype %s' % sr.d type) 479 # translate Python (0 ... N-1) into Fortran (1 ... N) with + 1 480 il, iu = sr + 1 ValueError: when using select="i", select_range must contain integers, got dtype int64

[nrosjat]

Same for me. I get the same error again after replacing my changes by the fix.

[davide-aloi]

@larsoner I think it worked for me but I know get this error: RuntimeError: Invalid Qt API 'pyqt5', valid values are: 'pyqt' or 'pyside'. I guess is not related with the previous issue though.

[larsoner]

Can you try our latest and greatest updated conda environment (merged today or yesterday):

https://github.com/mne-tools/mne-python/blob/master/environment.yml

[davide-aloi]

I have managed to build that environment in miniconda. I do not get any error now, but the mayavi window that should show the connectivity results just stucks as soon as it appears and crashes.

I am also struggling in creating the environment with the yml file, anaconda seems to stuck somewhere as some packages are not installed (as mne or all the packages listed after it, such as mayavi for example).

ps. I uninstalled and reinstalled anaconda and miniconda before doing it.

[davide-aloi]

Now using miniconda and ipython, even simple functions as raw.plot are not working, the windows are not responding and crashing. The raw.plot function is instead working if I use a python terminal, but I get an error when trying to do: "from mayavi import mlab" (from the sensor connectivity script). In Ipython I don't get this error, but the output windows are not working.

[larsoner]

Grrrr

Can you try pip install --upgrade --no-deps pyqt to see if this fixes it?

[davide-aloi]

"Could not find a version that satisfies the requirement pyqt (from versions: ) No matching distribution found for pyqt You are using pip version 9.0.1, however version 9.0.3 is available. You should consider upgrading via the 'python -m pip install --upgrade pip' comm and." :/

[larsoner]

Sorry, I meant pyqt5. Are you on gitter? We could iterate there more easily.

[larsoner]

https://gitter.im/mne-tools/mne-python

[davide-aloi]

After installing the pre-mentioned environment (https://github.com/mne-tools/mne-python/blob/master/environment.yml) I was still having problems with 3d visualization. The Mayavi window was not showing anything/freezing. I also get an error message from another window stating: "GL version 2.1 with the gpu_shader4 extension is not supported by your graphics driver but is required for the new OpenGL rendering backend. Please update your OpenGL driver. If you are using Mesa please make sure you have version 10.6.5 or later and make sure your driver in Mesa supports OpenGL 3.2.".

The issue was solved as suggested by Larsoner, using the %gui qt tweak and by making python using the NVIDIA card in Intel/NVIDIA switchable graphics feature as following: in the NVIDIA control panel IIRC you should set which programs use the GPU and add the python.exe from your env (e.g., C:\Users\whoever\miniconda3\envs\mne\bin\python.exe). You should add Ipython.exe and Spider.exe as well. That worked for me.

[agramfort]

Thanks for the feedback. This should be added to our install doc

[agramfort]

can you try to insert these info in this file:

https://raw.githubusercontent.com/mne-tools/mne-python/master/doc/install_mne_python.rst

I can take care of reviewing and pushing it in the mne doc later.

[larsoner]

Actually I think it would fit better in faq.rst, maybe with a clearer link from the main install_mne_python.rst. We don't want that install page to become the landing page for every possible system config / install problem.

[agramfort]

fine with me