pyms error

[j-moore94]

Hi, I installed mountainsort through the Ubuntu 16.04 app on Windows 10, and it seemed to install correctly, but then could not seem to run pyms processors. I am now getting this when I type mp-list-processors:

2018-11-23:17-47-03-107 mproc.processormanager warning "Potential problem with executable processor file: /opt/mountainlab/packages/pyms/basic/basic.mp. Expected json output but got empty string." 2018-11-23:17-47-03-112 mproc.processormanager warning "Executable processor file did not return output for spec: /opt/mountainlab/packages/pyms/basic/basic.mp" 2018-11-23:17-47-03-419 mproc.processormanager warning "Potential problem with executable processor file: /opt/mountainlab/packages/pyms/curation/curation.mp. Expected json output but got empty string." 2018-11-23:17-47-03-424 mproc.processormanager warning "Executable processor file did not return output for spec: /opt/mountainlab/packages/pyms/curation/curation.mp" 2018-11-23:17-47-03-732 mproc.processormanager warning "Potential problem with executable processor file: /opt/mountainlab/packages/pyms/synthesis/synthesis.mp. Expected json output but got empty string." 2018-11-23:17-47-03-737 mproc.processormanager warning "Executable processor file did not return output for spec: /opt/mountainlab/packages/pyms/synthesis/synthesis.mp" 2018-11-23:17-47-04-043 mproc.processormanager warning "Potential problem with executable processor file: /opt/mountainlab/packages/pyms/validation/validation.mp. Expected json output but got empty string." 2018-11-23:17-47-04-047 mproc.processormanager warning "Executable processor file did not return output for spec: /opt/mountainlab/packages/pyms/validation/validation.mp" 2018-11-23:17-47-04-351 mproc.processormanager warning "Potential problem with executable processor file: /opt/mountainlab/packages/pyms/basic/basic.mp. Expected json output but got empty string." 2018-11-23:17-47-04-355 mproc.processormanager warning "Executable processor file did not return output for spec: /opt/mountainlab/packages/pyms/basic/basic.mp" 2018-11-23:17-47-04-655 mproc.processormanager warning "Potential problem with executable processor file: /opt/mountainlab/packages/pyms/curation/curation.mp. Expected json output but got empty string." 2018-11-23:17-47-04-660 mproc.processormanager warning "Executable processor file did not return output for spec: /opt/mountainlab/packages/pyms/curation/curation.mp" 2018-11-23:17-47-04-961 mproc.processormanager warning "Potential problem with executable processor file: /opt/mountainlab/packages/pyms/synthesis/synthesis.mp. Expected json output but got empty string." 2018-11-23:17-47-04-966 mproc.processormanager warning "Executable processor file did not return output for spec: /opt/mountainlab/packages/pyms/synthesis/synthesis.mp" 2018-11-23:17-47-05-265 mproc.processormanager warning "Potential problem with executable processor file: /opt/mountainlab/packages/pyms/validation/validation.mp. Expected json output but got empty string." 2018-11-23:17-47-05-270 mproc.processormanager warning "Executable processor file did not return output for spec: /opt/mountainlab/packages/pyms/validation/validation.mp" banjoview.cross_correlograms mountainsortalg.ms3 mountainsortalg.ms3alg ms3.apply_timestamp_offset ms3.apply_whitening_matrix ms3.bandpass_filter ms3.cluster_metrics ms3.combine_cluster_metrics ms3.combine_firing_segments ms3.combine_firings ms3.compute_amplitudes ms3.compute_templates ms3.compute_whitening_matrix ms3.concat_event_times ms3.concat_firings ms3.concat_timeseries ms3.confusion_matrix ms3.create_firings ms3.create_multiscale_timeseries ms3.extract_clips ms3.extract_firings ms3.isolation_metrics ms3.link_segments ms3.load_test ms3.mask_out_artifacts ms3.mv_compute_amplitudes ms3.mv_compute_templates ms3.mv_discrimhist ms3.mv_extract_clips ms3.mv_extract_clips_features ms3.mv_subfirings ms3.reorder_labels ms3.run_metrics_script ms3.split_firings ms3.whiten ms3.whiten_clips spikeview.metrics1 spikeview.templates

Would really appreciate any help.

[wysota]

Would you please explain step by step how you installed mountainsort?

[j-moore94]

Thanks for the reply. Yes, I followed the instructions on https://mountainsort.readthedocs.io/en/latest/ to install with apt-get, and there didn't seem to be any issues with the main installation. I did encounter an issue with using pip3, so did "sudo apt-get install python3-numpy" instead - could this be causing the problem?

[wysota]

This is the old version of mountainsort that is not currently supported. Please follow instructions on the following page: https://github.com/flatironinstitute/mountainsort_examples/blob/master/README.md

[j-moore94]

Yeah, I actually need the old version. Do you know what the issue could be?

[wysota]

What happens if you run /opt/mountainlab/packages/pyms/basic/basic.mp spec?

[j-moore94]

I get this message: Traceback (most recent call last): File "/opt/mountainlab/packages/pyms/basic/basic.py", line 7, in from mlpy import ProcessorManager File "/opt/mountainlab/packages/pyms/mlpy/init.py", line 2, in from .processormanager import ProcessorManager File "/opt/mountainlab/packages/pyms/mlpy/processormanager.py", line 4, in from numpydoc import docscrape File "/usr/lib/python3/dist-packages/numpydoc/init.py", line 3, in from .numpydoc import setup File "/usr/lib/python3/dist-packages/numpydoc/numpydoc.py", line 29, in from sphinx.util.compat import Directive File "/usr/lib/python3/dist-packages/sphinx/util/compat.py", line 14, in from docutils.parsers.rst import Directive # noqa File "/usr/lib/python3/dist-packages/docutils/parsers/rst/init.py", line 75, in from docutils.parsers.rst import states File "/usr/lib/python3/dist-packages/docutils/parsers/rst/states.py", line 116, in from docutils.parsers.rst import directives, languages, tableparser, roles File "/usr/lib/python3/dist-packages/docutils/parsers/rst/roles.py", line 78, in from docutils.utils.code_analyzer import Lexer, LexerError File "/usr/lib/python3/dist-packages/docutils/utils/code_analyzer.py", line 13, in from pygments.lexers import get_lexer_by_name File "/usr/lib/python3/dist-packages/pygments/lexers/init.py", line 20, in from pygments.plugin import find_plugin_lexers File "/usr/lib/python3/dist-packages/pygments/plugin.py", line 39, in import pkg_resources File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2927, in @_call_aside File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2913, in _call_aside f(*args, **kwargs) File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2952, in _initialize_master_working_set add_activation_listener(lambda dist: dist.activate()) File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 956, in subscribe callback(dist) File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2952, in add_activation_listener(lambda dist: dist.activate()) File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2515, in activate declare_namespace(pkg) File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2097, in declare_namespace _handle_ns(packageName, path_item) File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2047, in _handle_ns _rebuild_mod_path(path, packageName, module) File "/usr/lib/python3/dist-packages/pkg_resources/init.py", line 2066, in _rebuild_mod_path orig_path.sort(key=position_in_sys_path) AttributeError: '_NamespacePath' object has no attribute 'sort'

[wysota]

Indeed it looks like a problem with your python installation.

Try running pip3 install --upgrade setuptools and reinstall numpy using pip3. If you have problems with that, post your errors.

[j-moore94]

Okay, yeah, thought it could be that. pip3 isn't working at all - I get this error when I run 'pip3 install --upgrade setuptools':

Traceback (most recent call last): File "/usr/lib/python3/dist-packages/pip/_vendor/init.py", line 33, in vendored import(vendored_name, globals(), locals(), level=0) ImportError: No module named 'pip._vendor.pkg_resources'

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/usr/bin/pip3", line 9, in from pip import main File "/usr/lib/python3/dist-packages/pip/init.py", line 13, in from pip.exceptions import InstallationError, CommandError, PipError File "/usr/lib/python3/dist-packages/pip/exceptions.py", line 6, in from pip._vendor.six import iteritems File "/usr/lib/python3/dist-packages/pip/_vendor/init.py", line 75, in vendored("pkg_resources") File "/usr/lib/python3/dist-packages/pip/_vendor/init.py", line 36, in vendored import(modulename, globals(), locals(), level=0) File "", line 969, in _find_and_load File "", line 958, in _find_and_load_unlocked File "", line 664, in _load_unlocked File "", line 634, in _load_backward_compatible File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 2927, in File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 2913, in _call_aside File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 2952, in _initialize_master_working_set File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 956, in subscribe File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 2952, in File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 2515, in activate File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 2097, in declare_namespace File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 2047, in _handle_ns File "/usr/share/python-wheels/pkg_resources-0.0.0-py2.py3-none-any.whl/pkg_resources/init.py", line 2066, in _rebuild_mod_path AttributeError: '_NamespacePath' object has no attribute 'sort'

[wysota]

You do have python3-pip and python3-setuptools installed, right?

Also, are you using a regular Python installation (apt install ...) or do you have some conda environment installed?

[j-moore94]

Yes, both installed, and I installed python3 using apt install python3. I did also have python installed through conda, but tried removing it and am still getting this issue.

[wysota]

Conda tends to break the default installation but if you had that issue before then it shouldn't matter. Maybe try setting up a virtualenv and perform the installation there.

Something along the lines of:

python3 -m venv .mlenv
. .mlenv/bin/activate
pip install ...
ml-list-processors

[j-moore94]

Okay, thanks, will give that a go. Should I re-install everything in the venv or just numpy?

[wysota]

venv gives you a clean python setup so you need to install everything. Best to simply repeat MountainSort installation while in the virtual env.

[j-moore94]

Just tried re-installing within the venv and still getting the original pyms error when I type mp-list-processors.

[wysota]

Well, the internet claims pip install --upgrade pip and pip install --upgrade setuptools should fix your issue (you might need to replace with pip3) so you can try that (so upgrade pip first and then setuptools). What you are dealing with is unrelated to MountainSort itself but rather to your Python installation. If all elses fail, you might want to try a newer Ubuntu version or at least a newer Python installation.

But in the end you should probably switch to more recent MountainSort release.

[j-moore94]

Okay, thank you for your help - it's good to know it is most likely that. I will try and find a solution.