aldro61 / kover

Learn interpretable computational phenotyping models from k-merized genomic data
http://aldro61.github.io/kover/
GNU General Public License v3.0
50 stars 14 forks source link

ImportError: No module named kover.dataset.create #37

Closed animesh closed 7 years ago

animesh commented 7 years ago

Dear Developers,

I am facing an ImportError: No module named kover.dataset.create while trying to run the following command:

bashs@promec1-vm:~/kover$ bin/kover dataset create from-reads --genomic-data gr.tsv --phenotype-name "Pitta" --phenotype-metadata md.tsv --output s2r.kover --progress Traceback (most recent call last): File "bin/kover", line 759, in CommandLineInterface() File "bin/kover", line 504, in init getattr(self, args.command)() File "bin/kover", line 524, in dataset getattr(dataset_tool, args.command)() File "bin/kover", line 244, in create getattr(creation_tool, args.datasource)() File "bin/kover", line 200, in from_reads from kover.dataset.create import from_reads ImportError: No module named kover.dataset.create

Below are contents of metadata files: bashs@promec1-vm:~/kover$ cat *.tsv S2 d1 S2r d2 S20-2r d2 gr.tsv

S2r 0 S2 1 S20-2r 0 md.tsv

bashs@promec1-vm:~/kover$ tree d? d1 └── S2 d2 ├── S20-2r └── S2r

Any ideas what might be the reason? Below are top couple of rows from the file: bashs@promec1-vm:~/kover$ head -n 2 d?/S2* ==> d1/S2 <== @@FCC5TW3ACXX:3:1101:1863:2087#TCCGTCTA/1 TCGGCGCGAGCGCCAGAATGAACGCGGGCATGCCGATGGTGAGAGCGCCGATATACGTGATATGGCGCGGCAGGTATGGGTATGAGATCGGAAGAGCACA ==> d2/S20-2r <== @@FCC5TW3ACXX:3:1101:1489:2068#TGGCTTCA/1 TCTATCCTTATCCTAATCTGTTGCCGGATGCCATCCATCCGAATGTGGAAGGTGCCGGCATCCTGGCGAAAACCGTTTATGGCGGTATTACAGGAAATTA ==> d2/S2r <== @@FCC5TW3ACXX:3:1101:1863:2087#TCCGTCTA/2 CATACCCATACCTGCCGCGCCATATCACGTATATCGGCGCTCTCACCATCGGCATGCCCGCGTTCATTCTGGCGCTCGCGCCGAAGATCGGAAGAGCGTC bashs@promec1-vm:~/kover$ head -n 2 d?/S2* ==> d1/S2 <== @@FCC5TW3ACXX:3:1101:1863:2087#TCCGTCTA/1 TCGGCGCGAGCGCCAGAATGAACGCGGGCATGCCGATGGTGAGAGCGCCGATATACGTGATATGGCGCGGCAGGTATGGGTATGAGATCGGAAGAGCACA ==> d2/S20-2r <== @@FCC5TW3ACXX:3:1101:1489:2068#TGGCTTCA/1 TCTATCCTTATCCTAATCTGTTGCCGGATGCCATCCATCCGAATGTGGAAGGTGCCGGCATCCTGGCGAAAACCGTTTATGGCGGTATTACAGGAAATTA ==> d2/S2r <== @@FCC5TW3ACXX:3:1101:1863:2087#TCCGTCTA/2 CATACCCATACCTGCCGCGCCATATCACGTATATCGGCGCTCTCACCATCGGCATGCCCGCGTTCATTCTGGCGCTCGCGCCGAAGATCGGAAGAGCGTC Looking forward to hear from you ;) Best regards, Ani
aldro61 commented 7 years ago

Hi Ani,

It seems that the installation of the kover core failed. Can you please run the install.sh script and reply with the output in an attached file?

Cheers, Alex

animesh commented 7 years ago

Dear Alex,

Thanks for the quick look and response. I am attaching the install-log-[0-2].txt files obtained via: animeshs@promec1-vm:~/kover$ ./install.sh --install-dir . 2>install-log-2.txt 1>install-log-1.txt > install-log-0.txt

hope this is what you meant?

Best regards,

Ani

--------------------------"The Answer Lies In The Genome"--------------------------

On Thu, Feb 9, 2017 at 2:35 PM, Alexandre Drouin notifications@github.com wrote:

Hi Ani,

It seems that the installation of the kover core failed. Can you please run the install.sh script and reply with the output in an attached file?

Cheers, Alex

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aldro61/kover/issues/37#issuecomment-278642193, or mute the thread https://github.com/notifications/unsubscribe-auth/AABT0QKcE6x8719Cb1470bgkutPURLk4ks5raxY_gaJpZM4L8F1_ .

██╗ ██╗ ██╗██╗ ██╗ ██╗███████╗██████╗ ██║ ██╔╝██╔╝╚██╗ ██║ ██║██╔════╝██╔══██╗ █████╔╝██╔╝ ╚██╗██║ ██║█████╗ ██████╔╝ ██╔═██╗╚██╗ ██╔╝╚██╗ ██╔╝██╔══╝ ██╔══██╗ ██║ ██╗╚██╗██╔╝ ╚████╔╝ ███████╗██║ ██║ ╚═╝ ╚═╝ ╚═╝╚═╝ ╚═══╝ ╚══════╝╚═╝ ╚═╝

By: Alexandre Drouin

Contributors:

Installing GATB

Installing setuptools

running install

Installing Kover

running install

Cleaning up... Creating Kover executable Done.

install.sh: 32: install.sh: cmake: not found make: *** No targets specified and no makefile found. Stop. Downloading https://pypi.python.org/packages/source/s/setuptools/setuptools-18.5.zip Extracting in /tmp/tmpLkjKvv Now working in /tmp/tmpLkjKvv/setuptools-18.5 Installing Setuptools error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the installation directory:

[Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/test-easy-install-36578.write-test'

The installation directory you specified (via --install-dir, --prefix, or the distutils default setting) was:

/usr/local/lib/python2.7/dist-packages/

Perhaps your account does not have write access to this directory? If the installation directory is a system-owned directory, you may need to sign in as the administrator or "root" account. If you do not have administrative access to this machine, you may wish to choose a different installation directory, preferably one that is listed in your PYTHONPATH environment variable.

For information on other options, you may wish to consult the documentation at:

https://pythonhosted.org/setuptools/easy_install.html

Please make the appropriate changes for your system and try again.

Something went wrong during the installation. See the error message above. error: can't create or remove files in install directory

The following error occurred while trying to add or remove files in the installation directory:

[Errno 13] Permission denied: '/usr/local/lib/python2.7/dist-packages/test-easy-install-36582.write-test'

The installation directory you specified (via --install-dir, --prefix, or the distutils default setting) was:

/usr/local/lib/python2.7/dist-packages/

Perhaps your account does not have write access to this directory? If the installation directory is a system-owned directory, you may need to sign in as the administrator or "root" account. If you do not have administrative access to this machine, you may wish to choose a different installation directory, preferably one that is listed in your PYTHONPATH environment variable.

For information on other options, you may wish to consult the documentation at:

https://pythonhosted.org/setuptools/easy_install.html

Please make the appropriate changes for your system and try again.

rm: cannot remove ‘build’: No such file or directory rm: cannot remove ‘dist’: No such file or directory rm: cannot remove ‘kover.egg-info’: No such file or directory

aldro61 commented 7 years ago

Hi Ani,

Thanks! Yes, this is what I meant.

Looking at the output, I see that the installation failed because the CMake program was not found. You can install it by following the instructions given here: https://cmake.org/install/.

The complete list of dependencies for Kover can be found here: http://aldro61.github.io/kover/doc_installation.html#prerequisites.

Once this is done, please re-run the installer and copy the output.

Cheers, Alex

animesh commented 7 years ago

Hi Alex,

Yes, sorry about it. In my defence, i missed that part, since the install-log-0.txt looked very clean :)

I think have managed to get cmake up and running:

animeshs@promec1-vm:~/kover$ cmake Usage

cmake [options] cmake [options]

Specify a source directory to (re-)generate a build system for it in the current working directory. Specify an existing build directory to re-generate its build system.

Run 'cmake --help' for more information.

However i am still facing the "ImportError: No module named kover.dataset.create" error.:

animeshs@promec1-vm:~/kover$ bin/kover dataset create from-reads --genomic-data gr.tsv --phenotype-name "Pitta" --phenotype-metadata md.tsv --output s2r.kover --progress Traceback (most recent call last): File "bin/kover", line 759, in CommandLineInterface() File "bin/kover", line 504, in init getattr(self, args.command)() File "bin/kover", line 524, in dataset getattr(dataset_tool, args.command)() File "bin/kover", line 244, in create getattr(creation_tool, args.datasource)() File "bin/kover", line 200, in from_reads from kover.dataset.create import from_reads ImportError: No module named kover.dataset.create

Hereby attached are the install logs.

Regards,

Ani

--------------------------"The Answer Lies In The Genome"--------------------------

On Thu, Feb 9, 2017 at 3:09 PM, Alexandre Drouin notifications@github.com wrote:

Hi Ani,

Thanks! Yes, this is what I meant.

Looking at the output, I see that the installation failed because the CMake program was not found. You can install it by following the instructions given here: https://cmake.org/install/.

The complete list of dependencies for Kover can be found here: http://aldro61.github.io/kover/doc_installation.html#prerequisites.

Once this is done, please re-run the installer and copy the output.

Cheers, Alex

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aldro61/kover/issues/37#issuecomment-278650889, or mute the thread https://github.com/notifications/unsubscribe-auth/AABT0aWrVVt9RgGOVBKkWJ4yYljpTFkAks5rax4igaJpZM4L8F1_ .

██╗ ██╗ ██╗██╗ ██╗ ██╗███████╗██████╗ ██║ ██╔╝██╔╝╚██╗ ██║ ██║██╔════╝██╔══██╗ █████╔╝██╔╝ ╚██╗██║ ██║█████╗ ██████╔╝ ██╔═██╗╚██╗ ██╔╝╚██╗ ██╔╝██╔══╝ ██╔══██╗ ██║ ██╗╚██╗██╔╝ ╚████╔╝ ███████╗██║ ██║ ╚═╝ ╚═╝ ╚═╝╚═╝ ╚═══╝ ╚══════╝╚═╝ ╚═╝

By: Alexandre Drouin

Contributors:

Installing GATB

Installing setuptools

running install running bdist_egg running egg_info writing requirements to setuptools.egg-info/requires.txt writing setuptools.egg-info/PKG-INFO writing top-level names to setuptools.egg-info/top_level.txt writing dependency_links to setuptools.egg-info/dependency_links.txt writing entry points to setuptools.egg-info/entry_points.txt reading manifest file 'setuptools.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'setuptools.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib.linux-x86_64-2.7 copying easy_install.py -> build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/setuptools copying setuptools/version.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/extension.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/site-patch.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/compat.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/py31compat.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/depends.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/windows_support.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/py27compat.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/init.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/py26compat.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/msvc9_support.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/package_index.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/archive_util.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/sandbox.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/unicode_utils.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/lib2to3_ex.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/ssl_support.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/utils.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/dist.py -> build/lib.linux-x86_64-2.7/setuptools creating build/lib.linux-x86_64-2.7/pkg_resources copying pkg_resources/init.py -> build/lib.linux-x86_64-2.7/pkg_resources creating build/lib.linux-x86_64-2.7/_markerlib copying _markerlib/init.py -> build/lib.linux-x86_64-2.7/_markerlib copying _markerlib/markers.py -> build/lib.linux-x86_64-2.7/_markerlib creating build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/sdist.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/bdist_wininst.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/setopt.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/upload_docs.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/egg_info.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/rotate.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/build_py.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/install_lib.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/install_scripts.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/easy_install.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/bdist_egg.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/build_ext.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/init.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/install.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/bdist_rpm.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/alias.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/install_egg_info.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/test.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/saveopts.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/develop.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/register.py -> build/lib.linux-x86_64-2.7/setuptools/command creating build/lib.linux-x86_64-2.7/pkg_resources/_vendor copying pkg_resources/_vendor/init.py -> build/lib.linux-x86_64-2.7/pkg_resources/_vendor creating build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging copying pkg_resources/_vendor/packaging/_compat.py -> build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging copying pkg_resources/_vendor/packaging/about.py -> build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging copying pkg_resources/_vendor/packaging/version.py -> build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging copying pkg_resources/_vendor/packaging/init.py -> build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging copying pkg_resources/_vendor/packaging/specifiers.py -> build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging copying pkg_resources/_vendor/packaging/_structures.py -> build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging copying setuptools/script (dev).tmpl -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/script.tmpl -> build/lib.linux-x86_64-2.7/setuptools creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/egg copying build/lib.linux-x86_64-2.7/easy_install.py -> build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/version.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/extension.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/site-patch.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/compat.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/py31compat.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/script (dev).tmpl -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/depends.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/windows_support.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/py27compat.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/script.tmpl -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/init.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/py26compat.py -> build/bdist.linux-x86_64/egg/setuptools creating build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/sdist.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/bdist_wininst.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/setopt.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/upload_docs.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/egg_info.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/rotate.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/build_py.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/install_lib.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/install_scripts.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/easy_install.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/bdist_egg.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/build_ext.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/init.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/install.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/bdist_rpm.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/alias.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/install_egg_info.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/test.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/saveopts.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/develop.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/register.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/msvc9_support.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/package_index.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/archive_util.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/sandbox.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/unicode_utils.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/lib2to3_ex.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/ssl_support.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/utils.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/dist.py -> build/bdist.linux-x86_64/egg/setuptools creating build/bdist.linux-x86_64/egg/pkg_resources copying build/lib.linux-x86_64-2.7/pkg_resources/init.py -> build/bdist.linux-x86_64/egg/pkg_resources creating build/bdist.linux-x86_64/egg/pkg_resources/_vendor copying build/lib.linux-x86_64-2.7/pkg_resources/_vendor/init.py -> build/bdist.linux-x86_64/egg/pkg_resources/_vendor creating build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging copying build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging/_compat.py -> build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging copying build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging/about.py -> build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging copying build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging/version.py -> build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging copying build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging/init.py -> build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging copying build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging/specifiers.py -> build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging copying build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging/_structures.py -> build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging creating build/bdist.linux-x86_64/egg/_markerlib copying build/lib.linux-x86_64-2.7/_markerlib/init.py -> build/bdist.linux-x86_64/egg/_markerlib copying build/lib.linux-x86_64-2.7/_markerlib/markers.py -> build/bdist.linux-x86_64/egg/_markerlib byte-compiling build/bdist.linux-x86_64/egg/easy_install.py to easy_install.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/version.py to version.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/extension.py to extension.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/site-patch.py to site-patch.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/compat.py to compat.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/py31compat.py to py31compat.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/depends.py to depends.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/windows_support.py to windows_support.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/py27compat.py to py27compat.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/init.py to init.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/py26compat.py to py26compat.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/sdist.py to sdist.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/bdist_wininst.py to bdist_wininst.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/setopt.py to setopt.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/upload_docs.py to upload_docs.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/egg_info.py to egg_info.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/rotate.py to rotate.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/build_py.py to build_py.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/install_lib.py to install_lib.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/install_scripts.py to install_scripts.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/easy_install.py to easy_install.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/bdist_egg.py to bdist_egg.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/build_ext.py to build_ext.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/init.py to init.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/install.py to install.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/bdist_rpm.py to bdist_rpm.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/alias.py to alias.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/install_egg_info.py to install_egg_info.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/test.py to test.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/saveopts.py to saveopts.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/develop.py to develop.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/register.py to register.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/msvc9_support.py to msvc9_support.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/package_index.py to package_index.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/archive_util.py to archive_util.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/sandbox.py to sandbox.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/unicode_utils.py to unicode_utils.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/lib2to3_ex.py to lib2to3_ex.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/ssl_support.py to ssl_support.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/utils.py to utils.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/dist.py to dist.pyc byte-compiling build/bdist.linux-x86_64/egg/pkg_resources/init.py to init.pyc byte-compiling build/bdist.linux-x86_64/egg/pkg_resources/_vendor/init.py to init.pyc byte-compiling build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging/_compat.py to _compat.pyc byte-compiling build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging/about.py to about.pyc byte-compiling build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging/version.py to version.pyc byte-compiling build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging/init.py to init.pyc byte-compiling build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging/specifiers.py to specifiers.pyc byte-compiling build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging/_structures.py to _structures.pyc byte-compiling build/bdist.linux-x86_64/egg/_markerlib/init.py to init.pyc byte-compiling build/bdist.linux-x86_64/egg/_markerlib/markers.py to markers.pyc creating build/bdist.linux-x86_64/egg/EGG-INFO copying setuptools.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO copying setuptools.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying setuptools.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying setuptools.egg-info/entry_points.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying setuptools.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying setuptools.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO copying setuptools.egg-info/zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO creating dist creating 'dist/setuptools-18.5-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it removing 'build/bdist.linux-x86_64/egg' (and everything under it) Processing setuptools-18.5-py2.7.egg Removing /usr/local/lib/python2.7/dist-packages/setuptools-18.5-py2.7.egg Copying setuptools-18.5-py2.7.egg to /usr/local/lib/python2.7/dist-packages setuptools 18.5 is already the active version in easy-install.pth Installing easy_install script to /usr/local/bin Installing easy_install-2.7 script to /usr/local/bin

Installed /usr/local/lib/python2.7/dist-packages/setuptools-18.5-py2.7.egg Processing dependencies for setuptools==18.5 Finished processing dependencies for setuptools==18.5

Installing Kover

running install running bdist_egg running egg_info creating kover.egg-info writing requirements to kover.egg-info/requires.txt writing kover.egg-info/PKG-INFO writing top-level names to kover.egg-info/top_level.txt writing dependency_links to kover.egg-info/dependency_links.txt writing manifest file 'kover.egg-info/SOURCES.txt' reading manifest file 'kover.egg-info/SOURCES.txt' writing manifest file 'kover.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/kover copying kover/init.py -> build/lib.linux-x86_64-2.7/kover copying kover/utils.py -> build/lib.linux-x86_64-2.7/kover creating build/lib.linux-x86_64-2.7/kover/learning copying kover/learning/init.py -> build/lib.linux-x86_64-2.7/kover/learning copying kover/learning/experiment.py -> build/lib.linux-x86_64-2.7/kover/learning creating build/lib.linux-x86_64-2.7/kover/dataset copying kover/dataset/create.py -> build/lib.linux-x86_64-2.7/kover/dataset copying kover/dataset/split.py -> build/lib.linux-x86_64-2.7/kover/dataset copying kover/dataset/init.py -> build/lib.linux-x86_64-2.7/kover/dataset copying kover/dataset/ds.py -> build/lib.linux-x86_64-2.7/kover/dataset creating build/lib.linux-x86_64-2.7/kover/learning/set_covering_machine copying kover/learning/set_covering_machine/init.py -> build/lib.linux-x86_64-2.7/kover/learning/set_covering_machine copying kover/learning/set_covering_machine/rules.py -> build/lib.linux-x86_64-2.7/kover/learning/set_covering_machine copying kover/learning/set_covering_machine/models.py -> build/lib.linux-x86_64-2.7/kover/learning/set_covering_machine copying kover/learning/set_covering_machine/scm.py -> build/lib.linux-x86_64-2.7/kover/learning/set_covering_machine creating build/lib.linux-x86_64-2.7/kover/dataset/tools copying kover/dataset/tools/init.py -> build/lib.linux-x86_64-2.7/kover/dataset/tools copying kover/dataset/tools/kmer_count.py -> build/lib.linux-x86_64-2.7/kover/dataset/tools copying kover/dataset/tools/kmer_pack.py -> build/lib.linux-x86_64-2.7/kover/dataset/tools running build_ext building 'kover/learning/set_covering_machine/popcount' extension creating build/temp.linux-x86_64-2.7 creating build/temp.linux-x86_64-2.7/kover creating build/temp.linux-x86_64-2.7/kover/learning creating build/temp.linux-x86_64-2.7/kover/learning/set_covering_machine x86_64-linux-gnu-gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.7 -I/usr/lib/python2.7/dist-packages/numpy/core/include -c kover/learning/set_covering_machine/popcount.c -o build/temp.linux-x86_64-2.7/kover/learning/set_covering_machine/popcount.o -march=native x86_64-linux-gnu-gcc -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -D_FORTIFY_SOURCE=2 -g -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security build/temp.linux-x86_64-2.7/kover/learning/set_covering_machine/popcount.o -o build/lib.linux-x86_64-2.7/kover/learning/set_covering_machine/popcount.so creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/kover copying build/lib.linux-x86_64-2.7/kover/init.py -> build/bdist.linux-x86_64/egg/kover creating build/bdist.linux-x86_64/egg/kover/learning creating build/bdist.linux-x86_64/egg/kover/learning/set_covering_machine copying build/lib.linux-x86_64-2.7/kover/learning/set_covering_machine/init.py -> build/bdist.linux-x86_64/egg/kover/learning/set_covering_machine copying build/lib.linux-x86_64-2.7/kover/learning/set_covering_machine/rules.py -> build/bdist.linux-x86_64/egg/kover/learning/set_covering_machine copying build/lib.linux-x86_64-2.7/kover/learning/set_covering_machine/models.py -> build/bdist.linux-x86_64/egg/kover/learning/set_covering_machine copying build/lib.linux-x86_64-2.7/kover/learning/set_covering_machine/popcount.so -> build/bdist.linux-x86_64/egg/kover/learning/set_covering_machine copying build/lib.linux-x86_64-2.7/kover/learning/set_covering_machine/scm.py -> build/bdist.linux-x86_64/egg/kover/learning/set_covering_machine copying build/lib.linux-x86_64-2.7/kover/learning/init.py -> build/bdist.linux-x86_64/egg/kover/learning copying build/lib.linux-x86_64-2.7/kover/learning/experiment.py -> build/bdist.linux-x86_64/egg/kover/learning copying build/lib.linux-x86_64-2.7/kover/utils.py -> build/bdist.linux-x86_64/egg/kover creating build/bdist.linux-x86_64/egg/kover/dataset copying build/lib.linux-x86_64-2.7/kover/dataset/create.py -> build/bdist.linux-x86_64/egg/kover/dataset creating build/bdist.linux-x86_64/egg/kover/dataset/tools copying build/lib.linux-x86_64-2.7/kover/dataset/tools/init.py -> build/bdist.linux-x86_64/egg/kover/dataset/tools copying build/lib.linux-x86_64-2.7/kover/dataset/tools/kmer_count.py -> build/bdist.linux-x86_64/egg/kover/dataset/tools copying build/lib.linux-x86_64-2.7/kover/dataset/tools/kmer_pack.py -> build/bdist.linux-x86_64/egg/kover/dataset/tools copying build/lib.linux-x86_64-2.7/kover/dataset/split.py -> build/bdist.linux-x86_64/egg/kover/dataset copying build/lib.linux-x86_64-2.7/kover/dataset/init.py -> build/bdist.linux-x86_64/egg/kover/dataset copying build/lib.linux-x86_64-2.7/kover/dataset/ds.py -> build/bdist.linux-x86_64/egg/kover/dataset byte-compiling build/bdist.linux-x86_64/egg/kover/init.py to init.pyc byte-compiling build/bdist.linux-x86_64/egg/kover/learning/set_covering_machine/init.py to init.pyc byte-compiling build/bdist.linux-x86_64/egg/kover/learning/set_covering_machine/rules.py to rules.pyc byte-compiling build/bdist.linux-x86_64/egg/kover/learning/set_covering_machine/models.py to models.pyc byte-compiling build/bdist.linux-x86_64/egg/kover/learning/set_covering_machine/scm.py to scm.pyc byte-compiling build/bdist.linux-x86_64/egg/kover/learning/init.py to init.pyc byte-compiling build/bdist.linux-x86_64/egg/kover/learning/experiment.py to experiment.pyc byte-compiling build/bdist.linux-x86_64/egg/kover/utils.py to utils.pyc byte-compiling build/bdist.linux-x86_64/egg/kover/dataset/create.py to create.pyc byte-compiling build/bdist.linux-x86_64/egg/kover/dataset/tools/init.py to init.pyc byte-compiling build/bdist.linux-x86_64/egg/kover/dataset/tools/kmer_count.py to kmer_count.pyc byte-compiling build/bdist.linux-x86_64/egg/kover/dataset/tools/kmer_pack.py to kmer_pack.pyc byte-compiling build/bdist.linux-x86_64/egg/kover/dataset/split.py to split.pyc byte-compiling build/bdist.linux-x86_64/egg/kover/dataset/init.py to init.pyc byte-compiling build/bdist.linux-x86_64/egg/kover/dataset/ds.py to ds.pyc creating stub loader for kover/learning/set_covering_machine/popcount.so byte-compiling build/bdist.linux-x86_64/egg/kover/learning/set_covering_machine/popcount.py to popcount.pyc installing package data to build/bdist.linux-x86_64/egg running install_data creating build/bdist.linux-x86_64/egg/kover/dataset/tools/kmer_tools

Cleaning up... Creating Kover executable Done.

-- gatb-core is here '/home/animeshs/kover/core/kover/dataset/tools/kmer_tools/thirdparty/gatb-core/gatb-core' -- CppUnit: found headers (in CPPUNIT_INCLUDE_DIR-NOTFOUND) but not the static library (CPPUNIT_INCLUDE_DIR-NOTFOUND/../lib/libcppunit.a) -- CppUnit NOT FOUND

-- C++ EXTENSIONS WILL BE USED...

-- SSE 4.2 detected

-- WILL COMPILE MPHF! (COMPILER VERSION IS HIGH ENOUGH)

-- Options: -O3 -DNDEBUG -Wall -Wno-unused-function -Wno-unused-variable -Wno-parentheses -Wno-format -Wno-strict-aliasing -Wno-array-bounds -Wno-unknown-pragmas -DINT128_FOUND -DWITH_LAMBDA_EXPRESSION -std=c++0x -DUSE_NEW_CXX -DWITH_MPHF -DEMPHF_USE_POPCOUNT=1 -Wno-invalid-offsetof -- OPTIMIZED KMER SIZES INTERVALS ARE 32 64 96 128 <-- max supported kmer size without recompilation CMake Warning at thirdparty/gatb-core/gatb-core/thirdparty/hdf5/CMakeLists.txt:562 (FIND_PACKAGE): Could not find a package configuration file provided by "ZLIB" with any of the following names:

ZLIBConfig.cmake
zlib-config.cmake

Add the installation prefix of "ZLIB" to CMAKE_PREFIX_PATH or set "ZLIB_DIR" to a directory containing one of the above files. If "ZLIB" provides a separate development package or SDK, be sure it has been installed.

CMake Error at thirdparty/gatb-core/gatb-core/thirdparty/hdf5/CMakeLists.txt:582 (MESSAGE): ZLib is Required for ZLib support in HDF5

make: *** No targets specified and no makefile found. Stop. Downloading https://pypi.python.org/packages/source/s/setuptools/setuptools-18.5.zip Extracting in /tmp/tmpactL67 Now working in /tmp/tmpactL67/setuptools-18.5 Installing Setuptools In file included from /usr/include/python2.7/numpy/ndarraytypes.h:1761:0, from /usr/include/python2.7/numpy/ndarrayobject.h:17, from /usr/include/python2.7/numpy/arrayobject.h:4, from kover/learning/set_covering_machine/popcount.c:242: /usr/include/python2.7/numpy/npy_1_7_deprecated_api.h:15:2: warning: #warning "Using deprecated NumPy API, disable it by " "#defining NPY_NO_DEPRECATED_API NPY_1_7_API_VERSION" [-Wcpp]

warning "Using deprecated NumPy API, disable it by " \

^ In file included from /usr/include/python2.7/numpy/ndarrayobject.h:26:0, from /usr/include/python2.7/numpy/arrayobject.h:4, from kover/learning/set_covering_machine/popcount.c:242: /usr/include/python2.7/numpy/__multiarray_api.h:1629:1: warning: ‘_import_array’ defined but not used [-Wunused-function] _import_array(void) ^ In file included from /usr/include/python2.7/numpy/ufuncobject.h:327:0, from kover/learning/set_covering_machine/popcount.c:243: /usr/include/python2.7/numpy/__ufunc_api.h:241:1: warning: ‘_import_umath’ defined but not used [-Wunused-function] _import_umath(void) ^ error: can't copy 'kover/dataset/tools/kmer_tools/build/bin/dsk2kover': doesn't exist or not a regular file rm: cannot remove ‘dist’: No such file or directory

aldro61 commented 7 years ago

Hi Ani,

Great, so now CMake is working properly. There is still an error in the building of GATB, which prevents the kover core from being installed. This is why you are getting the import errors when running kover.

Can you confirm that zlib is installed on your system? If you are using linux, you can install zlib by using: sudo apt-get install zlib1g-dev. This might be a dependency that we missed.

Best, Alex

animesh commented 7 years ago

OK :)

On Thu, Feb 9, 2017 at 4:27 PM, Alexandre Drouin notifications@github.com wrote:

sudo apt-get install zlib1g-dev

animeshs@promec1-vm:~/kover$ sudo apt-get install zlib1g-dev Reading package lists... Done Building dependency tree Reading state information... Done Package zlib1g-dev is not available, but is referred to by another package. This may mean that the package is missing, has been obsoleted, or is only available from another source

E: Package 'zlib1g-dev' has no installation candidate

However,

animeshs@promec1-vm:~/kover$ sudo apt-get install zlib1g Reading package lists... Done Building dependency tree Reading state information... Done zlib1g is already the newest version.

So probably the right repo line is missing? Which one do you suggest?

Regards,

Ani

--------------------------"The Answer Lies In The Genome"--------------------------

aldro61 commented 7 years ago

Allright, so this tells us that zlib is installed, but not it's development header, which are required by GATB. The key to solving your issue will be to find a way to install zlib1g-dev on your system. Which Linux distribution are you using?

animesh commented 7 years ago

animeshs@promec1-vm:~/kover$ uname -a Linux promec1-vm 3.19.0-79-generic #87~14.04.1-Ubuntu SMP Wed Dec 21 18:12:31 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

Regards,

Ani

--------------------------"The Answer Lies In The Genome"--------------------------

On Thu, Feb 9, 2017 at 5:22 PM, Alexandre Drouin notifications@github.com wrote:

Allright, so this tells us that zlib is installed, but not it's development header, which are required by GATB. The key to solving your issue will be to find a way to install zlib1g-dev on your system. Which Linux distribution are you using?

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aldro61/kover/issues/37#issuecomment-278692232, or mute the thread https://github.com/notifications/unsubscribe-auth/AABT0YrIfKcYfA0QHZGvfNgx-6mG1zRHks5raz1PgaJpZM4L8F1_ .

aldro61 commented 7 years ago

Thank you.

It is quite strange that the zlib1g-dev package is not found by apt-get. Can you run the following commands: sudo apt-get update; sudo apt-get install zlib1g-dev?

animesh commented 7 years ago

On Thu, Feb 9, 2017 at 6:01 PM, Alexandre Drouin notifications@github.com wrote:

sudo apt-get update; sudo apt-get install zlib1g-dev

That worked :) Should i re-run the intall script and test kover or some more deps to cover?

Regards,

Ani

--------------------------"The Answer Lies In The Genome"--------------------------

aldro61 commented 7 years ago

Great!

Yes, please re-run the installer. If the following command works: python -c "import kover", then kover should run fine. I would also recommend that you run the example on the documentation website to make sure that everything is working correctly.

animesh commented 7 years ago

Seems like we are heading somewhere as error has changed, something called h5py is missing:

animeshs@promec1-vm:~/kover$ bin/kover dataset create from-reads --genomic-data gr.tsv --phenotype-name "Pitta" --phenotype-metadata md.tsv --output s2r.kover --progress Traceback (most recent call last): File "bin/kover", line 759, in CommandLineInterface() File "bin/kover", line 504, in init getattr(self, args.command)() File "bin/kover", line 524, in dataset getattr(dataset_tool, args.command)() File "bin/kover", line 244, in create getattr(creation_tool, args.datasource)() File "bin/kover", line 200, in from_reads from kover.dataset.create import from_reads File "/usr/local/lib/python2.7/dist-packages/kover-1.2.1-py2.7-linux-x86_64.egg/kover/dataset/init.py", line 20, in from ds import KoverDataset File "/usr/local/lib/python2.7/dist-packages/kover-1.2.1-py2.7-linux-x86_64.egg/kover/dataset/ds.py", line 22, in from ..utils import _hdf5_open_no_chunk_cache File "/usr/local/lib/python2.7/dist-packages/kover-1.2.1-py2.7-linux-x86_64.egg/kover/utils.py", line 20, in import h5py as h ImportError: No module named h5py

Attached install logs. Probably this needs to be fixed before i venture into example set?

Regards,

Ani

--------------------------"The Answer Lies In The Genome"--------------------------

On Thu, Feb 9, 2017 at 6:44 PM, Alexandre Drouin notifications@github.com wrote:

Great!

Yes, please re-run the installer. If the following command works: python -c "import kover", then kover should run fine. I would also recommend that you run the example on the documentation website to make sure that everything is working correctly.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aldro61/kover/issues/37#issuecomment-278716917, or mute the thread https://github.com/notifications/unsubscribe-auth/AABT0c6qnOfdtwg8Buo6Q9O4kmhcR3Yiks5ra1BlgaJpZM4L8F1_ .

aldro61 commented 7 years ago

Hi Ani,

I don't see the attached installed logs.

The h5py Python package is a dependency of kover. It should have been installed automatically by the installer. If not, you can install it yourself: http://docs.h5py.org/en/latest/build.html.

Alex

animesh commented 7 years ago

Dear Alex,

Sorry for missing out the attachments. You were right about h5py was not able to install automagically because libhdf5-dev was missing, so following series of command fixed the error it seems (attached install logs):

sudo apt-get install python-dev sudo apt-get install libhdf5-dev sudo pip install h5py

So a new set of problem appears:

bin/kover dataset create from-reads --genomic-data gr.tsv --phenotype-name "Pitta" --phenotype-metadata md.tsv --output s2r.kover --progress Traceback (most recent call last): File "bin/kover", line 759, in CommandLineInterface() File "bin/kover", line 504, in init getattr(self, args.command)() File "bin/kover", line 524, in dataset getattr(dataset_tool, args.command)() File "bin/kover", line 244, in create getattr(creation_tool, args.datasource)() File "bin/kover", line 218, in from_reads progress=args.progress) File "/usr/local/lib/python2.7/dist-packages/kover-1.2.1-py2.7-linux-x86_64.egg/kover/dataset/create.py", line 375, in from_reads reads_folder_by_genome_id = dict(l.split() for l in open(reads_folders_list_path, "r")) ValueError: dictionary update sequence element #3 has length 0; 2 is required

It looks like error in my input, thus i would request if you could also take a look in my initial email with input briefly defined and suggest some solutions?

Regards,

Ani

--------------------------"The Answer Lies In The Genome"--------------------------

On Fri, Feb 10, 2017 at 3:18 PM, Alexandre Drouin notifications@github.com wrote:

Hi Ani,

I don't see the attached installed logs.

The h5py Python package is a dependency of kover. It should have been installed automatically by the installer. If not, you can install it yourself: http://docs.h5py.org/en/latest/build.html.

Alex

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aldro61/kover/issues/37#issuecomment-278952252, or mute the thread https://github.com/notifications/unsubscribe-auth/AABT0WCPNcBlrvRNhoTU7DKZLlSiyfHmks5rbHHEgaJpZM4L8F1_ .

██╗ ██╗ ██╗██╗ ██╗ ██╗███████╗██████╗ ██║ ██╔╝██╔╝╚██╗ ██║ ██║██╔════╝██╔══██╗ █████╔╝██╔╝ ╚██╗██║ ██║█████╗ ██████╔╝ ██╔═██╗╚██╗ ██╔╝╚██╗ ██╔╝██╔══╝ ██╔══██╗ ██║ ██╗╚██╗██╔╝ ╚████╔╝ ███████╗██║ ██║ ╚═╝ ╚═╝ ╚═╝╚═╝ ╚═══╝ ╚══════╝╚═╝ ╚═╝

By: Alexandre Drouin

Contributors:

Installing GATB

Installing setuptools

running install running bdist_egg running egg_info writing requirements to setuptools.egg-info/requires.txt writing setuptools.egg-info/PKG-INFO writing top-level names to setuptools.egg-info/top_level.txt writing dependency_links to setuptools.egg-info/dependency_links.txt writing entry points to setuptools.egg-info/entry_points.txt reading manifest file 'setuptools.egg-info/SOURCES.txt' reading manifest template 'MANIFEST.in' writing manifest file 'setuptools.egg-info/SOURCES.txt' installing library code to build/bdist.linux-x86_64/egg running install_lib running build_py creating build creating build/lib.linux-x86_64-2.7 copying easy_install.py -> build/lib.linux-x86_64-2.7 creating build/lib.linux-x86_64-2.7/setuptools copying setuptools/version.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/extension.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/site-patch.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/compat.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/py31compat.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/depends.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/windows_support.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/py27compat.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/init.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/py26compat.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/msvc9_support.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/package_index.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/archive_util.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/sandbox.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/unicode_utils.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/lib2to3_ex.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/ssl_support.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/utils.py -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/dist.py -> build/lib.linux-x86_64-2.7/setuptools creating build/lib.linux-x86_64-2.7/pkg_resources copying pkg_resources/init.py -> build/lib.linux-x86_64-2.7/pkg_resources creating build/lib.linux-x86_64-2.7/_markerlib copying _markerlib/init.py -> build/lib.linux-x86_64-2.7/_markerlib copying _markerlib/markers.py -> build/lib.linux-x86_64-2.7/_markerlib creating build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/sdist.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/bdist_wininst.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/setopt.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/upload_docs.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/egg_info.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/rotate.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/build_py.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/install_lib.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/install_scripts.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/easy_install.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/bdist_egg.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/build_ext.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/init.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/install.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/bdist_rpm.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/alias.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/install_egg_info.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/test.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/saveopts.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/develop.py -> build/lib.linux-x86_64-2.7/setuptools/command copying setuptools/command/register.py -> build/lib.linux-x86_64-2.7/setuptools/command creating build/lib.linux-x86_64-2.7/pkg_resources/_vendor copying pkg_resources/_vendor/init.py -> build/lib.linux-x86_64-2.7/pkg_resources/_vendor creating build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging copying pkg_resources/_vendor/packaging/_compat.py -> build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging copying pkg_resources/_vendor/packaging/about.py -> build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging copying pkg_resources/_vendor/packaging/version.py -> build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging copying pkg_resources/_vendor/packaging/init.py -> build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging copying pkg_resources/_vendor/packaging/specifiers.py -> build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging copying pkg_resources/_vendor/packaging/_structures.py -> build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging copying setuptools/script (dev).tmpl -> build/lib.linux-x86_64-2.7/setuptools copying setuptools/script.tmpl -> build/lib.linux-x86_64-2.7/setuptools creating build/bdist.linux-x86_64 creating build/bdist.linux-x86_64/egg copying build/lib.linux-x86_64-2.7/easy_install.py -> build/bdist.linux-x86_64/egg creating build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/version.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/extension.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/site-patch.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/compat.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/py31compat.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/script (dev).tmpl -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/depends.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/windows_support.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/py27compat.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/script.tmpl -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/init.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/py26compat.py -> build/bdist.linux-x86_64/egg/setuptools creating build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/sdist.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/bdist_wininst.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/setopt.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/upload_docs.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/egg_info.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/rotate.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/build_py.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/install_lib.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/install_scripts.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/easy_install.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/bdist_egg.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/build_ext.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/init.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/install.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/bdist_rpm.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/alias.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/install_egg_info.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/test.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/saveopts.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/develop.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/command/register.py -> build/bdist.linux-x86_64/egg/setuptools/command copying build/lib.linux-x86_64-2.7/setuptools/msvc9_support.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/package_index.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/archive_util.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/sandbox.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/unicode_utils.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/lib2to3_ex.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/ssl_support.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/utils.py -> build/bdist.linux-x86_64/egg/setuptools copying build/lib.linux-x86_64-2.7/setuptools/dist.py -> build/bdist.linux-x86_64/egg/setuptools creating build/bdist.linux-x86_64/egg/pkg_resources copying build/lib.linux-x86_64-2.7/pkg_resources/init.py -> build/bdist.linux-x86_64/egg/pkg_resources creating build/bdist.linux-x86_64/egg/pkg_resources/_vendor copying build/lib.linux-x86_64-2.7/pkg_resources/_vendor/init.py -> build/bdist.linux-x86_64/egg/pkg_resources/_vendor creating build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging copying build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging/_compat.py -> build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging copying build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging/about.py -> build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging copying build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging/version.py -> build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging copying build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging/init.py -> build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging copying build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging/specifiers.py -> build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging copying build/lib.linux-x86_64-2.7/pkg_resources/_vendor/packaging/_structures.py -> build/bdist.linux-x86_64/egg/pkg_resources/_vendor/packaging creating build/bdist.linux-x86_64/egg/_markerlib copying build/lib.linux-x86_64-2.7/_markerlib/init.py -> build/bdist.linux-x86_64/egg/_markerlib copying build/lib.linux-x86_64-2.7/_markerlib/markers.py -> build/bdist.linux-x86_64/egg/_markerlib byte-compiling build/bdist.linux-x86_64/egg/easy_install.py to easy_install.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/version.py to version.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/extension.py to extension.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/site-patch.py to site-patch.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/compat.py to compat.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/py31compat.py to py31compat.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/depends.py to depends.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/windows_support.py to windows_support.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/py27compat.py to py27compat.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/init.py to init.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/py26compat.py to py26compat.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/sdist.py to sdist.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/bdist_wininst.py to bdist_wininst.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/setopt.py to setopt.pyc byte-compiling build/bdist.linux-x86_64/egg/setuptools/command/upload_docs.py to upload_docs.pyc byte-compiling build

aldro61 commented 7 years ago

Hi Ani,

I think that the format of your input is incorrect. When using the from-reads option, the genomic data should be a file containing a single line per example, where the line has the format GENOMEID{TAB}FOLDER, where FOLDER is the folder containing the reads (fastq format) for the example. For more details: http://aldro61.github.io/kover/doc_input_formats.html#reads.

Also, it seems like you have only 3 examples in your dataset. To use Kover (or any machine learning algorithm), you need more examples than that. Depending on the complexity of what you are trying to model, you would need at least 50 examples (and that not a lot) to get an accurate model.

Alex

animesh commented 7 years ago

HI Alex,

I have recently started working with a metagenomic data, so pardon my naïve questions :) In total, we have about 20 samples sequenced using Illumina HiSeq 2500 platform. On an average 2-3 GB data, ~20 million paired end reads of 100 length per sample was generated.

So far I have managed to assemble these reads into a genome. These samples can be grouped into 3 categories based on their case histories and I would like to use your approach to check if there is any k-mer which can robustly discriminate these profiles based on read and/or assembled genome.

Currently the run was just to see if the pipeline works for pairwise comparison of two groups [called 0 and 1 here]. So I put the fastq files [S2, its paired version S2-r and S20-2r without its forward pair] in d[1-2] folder with the attached tsv as the directive.

So if i got it right, i should reverse the column in the tsv's to make sense for testing purpose?

Best regards,

Ani

--------------------------"The Answer Lies In The Genome"--------------------------

On Mon, Feb 13, 2017 at 2:12 AM, Alexandre Drouin notifications@github.com wrote:

Hi Ani,

I think that the format of your input is incorrect. When using the from-reads option, the genomic data should be a file containing a single line per example, where the line has the format GENOMEID{TAB}FOLDER, where FOLDER is the folder containing the reads (fastq format) for the example. For more details: http://aldro61.github.io/kover/doc_input_formats.html# reads.

Also, it seems like you have only 3 examples in your dataset. To use Kover (or any machine learning algorithm), you need more examples than that. Depending on the complexity of what you are trying to model, you would need at least 50 examples (and that not a lot) to get an accurate model.

Alex

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aldro61/kover/issues/37#issuecomment-279272507, or mute the thread https://github.com/notifications/unsubscribe-auth/AABT0ZkjurgSHjOAr0lsqI_gNFjWe5lQks5rb64KgaJpZM4L8F1_ .

aldro61 commented 7 years ago

Hi Ani,

Excuse me for the delay in responding.

1) If I have more than 2 groups, can I still use Kover? Yes, but you will need to consider multiple binary classification problems. You can use the _one versus rest or one versus one_ methods. I believe this was your plan.

2) What is the format of the TSV file passed as --genomic-data argument in kover dataset create? You are using the from-reads option, so this file should contain a single line per genome, with the following format GENOMEID{TAB}DIRECTORY, where directory is the path to a directory containing all the read files for this genome. If you have more than one read file per genome, do not create multiple lines with the same GENOMEID. Simply put all the read files in DIRECTORY. By looking at your TSV file, it seems that there are two genomes with the same value for DIRECTORY. This does not make sense, since it creates two identical learning examples.

So suppose that I have a genome with two read files (R1.fastq, R2.fastq), I would create a directory called Genome1_reads, with structure:

Genome1_reads:
    - R1.fastq
    - R2.fastq

and then add the following line to the TSV file Genome1 Genome1_reads.

3) Other note: If you try to test the pipeline using only 3 learning examples, you will likely run into issues with Kover. For example, the split command will likely fail, since there will not be enough examples to create cross-validation folds. Also, when you run kover learn, you risk not having any examples of a class in the training or testing set, which should be avoided. I would definitely recommend testing with more examples, possibly all 20.

I hope this solves you issues and that you'll get interesting results!

Cheers, Alex

animesh commented 7 years ago

Thanks Alex, better late than never :) Sorry about messing up the input parameters and yes i was planning to go for one-on-one for the 3-class. I did try to re-do the inputs following your outlined scheme, but hit an error:

bin/kover dataset create from-reads --genomic-data gr.tsv --phenotype-name "Pitta" --phenotype-metadata md.tsv --output s.kover --progress Traceback (most recent call last): File "bin/kover", line 759, in CommandLineInterface() File "bin/kover", line 504, in init getattr(self, args.command)() File "bin/kover", line 524, in dataset getattr(dataset_tool, args.command)() File "bin/kover", line 244, in create getattr(creation_tool, args.datasource)() File "bin/kover", line 218, in from_reads progress=args.progress) File "/usr/local/lib/python2.7/dist-packages/kover-1.2.1-py2.7-linux-x86_64.egg/kover/dataset/create.py", line 375, in from_reads reads_folder_by_genome_id = dict(l.split() for l in open(reads_folders_list_path, "r")) ValueError: dictionary update sequence element #4 has length 0; 2 is required

which looks like the issue with read files? Currently below is the structure of input genome read directories:

g1 ├── S2.fastq └── S2r.fastq g2 ├── s13.fastq └── s13-r.fastq g3 ├── S5.fastq └── S5-r.fastq g4 ├── S20-2.fastq └── S20-2r.fastq

and the tab separated params:

cat md.tsv g1 0 g2 1 g3 0 g4 1

cat gr.tsv S2 g1 s13 g2 S20 g4 S5 g3

any suggestions to proceed will be appreciated :)

Regards,

Ani

--------------------------"The Answer Lies In The Genome"--------------------------

On Thu, Feb 16, 2017 at 4:05 AM, Alexandre Drouin notifications@github.com wrote:

Hi Ani,

Excuse me for the delay in responding.

1.

If I have more than 2 groups, can I still use Kover? Yes, but you will need to consider multiple binary classification problems. You can use the one versus res https://en.wikipedia.org/wiki/Multiclass_classification#One-vs.-restt or one versus one https://en.wikipedia.org/wiki/Multiclass_classification#One-vs.-one methods. I believe this was your plan. 2.

What is the format of the TSV file passed as --genomic-data argument in kover dataset create? You are using the from-reads option, so this file should contain a single line per genome, with the following format GENOMEID{TAB}DIRECTORY, where directory is the path to a directory containing all the read files for this genome. If you have more than one read file per genome, do not create multiple lines with the same GENOMEID. Simply put all the read files in DIRECTORY. By looking at your TSV file, it seems that there are two genomes with the same value for DIRECTORY. This does not make sense, since it creates two identical learning examples.

So suppose that I have a genome with two read files (R1.fastq, R2.fastq), I would create a directory called Genome1_reads, with structure:

Genome1_reads:

  • R1.fastq
  • R2.fastq

and then add the following line to the TSV file Genome1 Genome1_reads.

  1. Other note: If you try to test the pipeline using only 3 learning examples, you will likely run into issues with Kover. For example, the split command will likely fail, since there will not be enough examples to create cross-validation folds. Also, when you run kover learn, you risk not having any examples of a class in the training or testing set, which should be avoided. I would definitely recommend testing with more examples, possibly all 20.

I hope this solves you issues and that you'll get interesting results!

Cheers, Alex

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aldro61/kover/issues/37#issuecomment-280217590, or mute the thread https://github.com/notifications/unsubscribe-auth/AABT0U6kv6z_8aBbjMlQSCXJMNHdEB1fks5rc70CgaJpZM4L8F1_ .

aldro61 commented 7 years ago

Hi Ani,

The first column in the gr.tsv file represents the genome identifiers. Same thing for the md.tsv. So the first column of both files should match. Basically, your md.tsv file is wrong. The first column is giving the read directory rather than the genome identifier.

If gr.tsv is:

S2      g1
s13     g2
S20     g4
S5      g3

then md.tsv should be:

S1      0
s12      1
S5      0
S20      1

This is documented here: http://aldro61.github.io/kover/doc_input_formats.html.

Alex

animesh commented 7 years ago

Thanks again Alex for keeping up with my stupidity :) I did implement the change

gr.tsv S2 g1 s13 g2 S20 g4 S5 g3

md.tsv S2 0 s13 1 S20 1 S5 0

however i am still facing following error

bin/kover dataset create from-reads --genomic-data gr.tsv --phenotype-name "Pita" --phenotype-metadata md.tsv --output ss.kover --progress Traceback (most recent call last): File "bin/kover", line 759, in CommandLineInterface() File "bin/kover", line 504, in init getattr(self, args.command)() File "bin/kover", line 524, in dataset getattr(dataset_tool, args.command)() File "bin/kover", line 244, in create getattr(creation_tool, args.datasource)() File "bin/kover", line 218, in from_reads progress=args.progress) File "/usr/local/lib/python2.7/dist-packages/kover-1.2.1-py2.7-linux-x86_64.egg/kover/dataset/create.py", line 375, in from_reads reads_folder_by_genome_id = dict(l.split() for l in open(reads_folders_list_path, "r")) ValueError: dictionary update sequence element #4 has length 0; 2 is required

does the file name within the directory has to follow genome_id too?

g1 ├── S2.fastq └── S2r.fastq

0 directories, 2 files g2 ├── s13.fastq └── s13-r.fastq

0 directories, 2 files g3 ├── S5.fastq └── S5-r.fastq

0 directories, 2 files g4 ├── S20-2.fastq └── S20-2r.fastq

or there is something wrong with fastq files itself?

==> g3/S5.fastq <== @@FCC5TW3ACXX:3:1101:1458:2045#TGAAGAGA/1 TGCATCCCCAAACAGCAGTAAACTGCATTGGCACGTTCTGGTCGGTCTTGCGGATAGTCTTGTAGAGAGCCTGGAGTGTCAAGAGATCGGAAGAGCACAC

==> g3/S5-r.fastq <== @@FCC5TW3ACXX:3:1101:1458:2045#TGAAGAGA/2 CTTGACACTCCAGGCTCTCTACAAGACTATCCGCAAGACCGACCAGAACGTGCCAATGCAGTTTACTGCTGTTTGGGGATGCAAGATCGGAAGAGCGTCG

==> g4/S20-2.fastq <== @@FCC5TW3ACXX:3:1101:1489:2068#TGGCTTCA/1 TCTATCCTTATCCTAATCTGTTGCCGGATGCCATCCATCCGAATGTGGAAGGTGCCGGCATCCTGGCGAAAACCGTTTATGGCGGTATTACAGGAAATTA

==> g4/S20-2r.fastq <== @@FCC5TW3ACXX:3:1101:1489:2068#TGGCTTCA/1 TCTATCCTTATCCTAATCTGTTGCCGGATGCCATCCATCCGAATGTGGAAGGTGCCGGCATCCTGGCGAAAACCGTTTATGGCGGTATTACAGGAAATTA

Or there is some other issue, any ideas?

Regards,

Ani

--------------------------"The Answer Lies In The Genome"--------------------------

On Thu, Feb 16, 2017 at 7:24 PM, Alexandre Drouin notifications@github.com wrote:

Hi Ani,

The first column in the gr.tsv file represents the genome identifiers. Same thing for the md.tsv. So the first column of both files should match. Basically, your md.tsv file is wrong. The first column is giving the read directory rather than the genome identifier.

If gr.tsv is:

S2 g1 s13 g2 S20 g4 S5 g3

then md.tsv should be:

S1 0 s12 1 S5 0 S20 1

This is documented here: http://aldro61.github.io/ kover/doc_input_formats.html.

Alex

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aldro61/kover/issues/37#issuecomment-280415098, or mute the thread https://github.com/notifications/unsubscribe-auth/AABT0c4vw87xFYyaOChL8rYaaKz98uRlks5rdJR2gaJpZM4L8F1_ .

aldro61 commented 7 years ago

Would you happen to have a blank line at the end of your gr.tsv file?

The parser is complaining because it encounters a line that doesn't have the GENOMEID{TAB}DIRECTORY format. Since the file you showed me has 4 lines and the error refers to line 4 (starting from 0), I suspect that a blank line is the issue.

animesh commented 7 years ago

Spot on Alex:

bin/kover dataset create from-reads --genomic-dat a gr.tsv --phenotype-name "Pita" --phenotype-metadata md.tsv --output ss.kover --progress

multidsk [==================================================] 100%

dsk2kover [==================================================] 100%

Thanks a lot for all the help!

Have a nice weekend :)

Regards,

Ani

--------------------------"The Answer Lies In The Genome"--------------------------

On Sat, Feb 18, 2017 at 4:17 PM, Alexandre Drouin notifications@github.com wrote:

Would you happen to have a blank line at the end of your gr.tsv file?

The parser is complaining because it encounters a line that doesn't have the GENOMEID{TAB}DIRECTORY format. Since the file you showed me has 4 lines and the error refers to line 4 (starting from 0), I suspect that a blank line is the issue.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/aldro61/kover/issues/37#issuecomment-280851827, or mute the thread https://github.com/notifications/unsubscribe-auth/AABT0ec-ETGoXX5Lk3TXZBAKWKz8-byTks5rdwuCgaJpZM4L8F1_ .

aldro61 commented 7 years ago

Great! I'm glad that your issue is resolved.

Have a nice weekend too!

Alex