Syntax issues with python

[shaman-narayanasamy]

Dear authors,

I am attempting to install iRep on my Macbook Pro (OSX Sierra), it runs python 2.7 and pip 9.0.1 for the corresponding version of python.

I followed the installation instructions, but I receive the following error when attempting to run the program.

$ iRep
Traceback (most recent call last):
  File "/usr/local/bin/iRep", line 19, in <module>
    import iRep.iRep as iRep
  File "/usr/local/lib/python2.7/site-packages/iRep/iRep.py", line 177
    print('# plotting data', file=sys.stderr)
                                 ^

However, iRep_filter.py seems to work fine:

$ iRep_filter.py -h
usage: iRep_filter.py [-h] -t [T [T ...]] [-c C] [-w W] [-r R] [-f F] [-g G]
                      [--long]

# combine and/or filter iRep.py output

optional arguments:
  -h, --help      show this help message and exit
  -t [T [T ...]]  iRep table(s)
  -c C            min. coverage (default = 5)
  -w W            min. percent windows passing filter (default = 98)
  -r R            min. r^2 value for fitting (default = 0.90)
  -f F            max. fragments/Mbp (default = 175)
  -g G            max. GC bias (default = no filter)
  --long          print in long format

Am I doing something wrong? Looking forward to hearing from you.

Best regards, Shaman

[christophertbrown]

Hi Shaman, iRep requires python 3. I recommend using 'pyenv' to manage python versions on your computer and to install python 3, and then re-install iRep using 'pip.' Hopefully this will fix the issue, but please do let me know if you are still having problems. Best, Chris

On Sep 2, 2017, at 5:58 AM, Shaman notifications@github.com wrote:

Dear authors,

I am attempting to install iRep on my Macbook Pro (OSX Sierra), it runs python 2.7 and pip 9.0.1 for the corresponding version of python.

I followed the installation instructions, but I receive the following error when attempting to run the program.

$ iRep Traceback (most recent call last): File "/usr/local/bin/iRep", line 19, in import iRep.iRep as iRep File "/usr/local/lib/python2.7/site-packages/iRep/iRep.py", line 177 print('# plotting data', file=sys.stderr)

However, `iRep_filter.py` seems to work fine:

$ iRep_filter.py -h
usage: iRep_filter.py [-h] -t [T [T ...]] [-c C] [-w W] [-r R] [-f F] [-g G]
[--long]

combine and/or filter iRep.py output

optional arguments:
-h, --help show this help message and exit
-t [T [T ...]] iRep table(s)
-c C min. coverage (default = 5)
-w W min. percent windows passing filter (default = 98)
-r R min. r^2 value for fitting (default = 0.90)
-f F max. fragments/Mbp (default = 175)
-g G max. GC bias (default = no filter)
--long print in long format

Am I doing something wrong? Looking forward to hearing from you.

Best regards,
Shaman
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub, or mute the thread.

[shaman-narayanasamy]

Hi Chris,

Thanks very much for the reply. I will attempt to install it the way you suggested and get back to you on that.

Best, Shaman

[shaman-narayanasamy]

Hi Chris,

A colleague of mine managed to install it on our cluster. It also worked for me when I tested in on a small dataset (three sam files, one genome). However, when I tried to run it on a large dataset (53 SAM files and 220 genomes), it seems to halt giving the following:

# parsing mapping files
# calculating coverage over sliding windows
Traceback (most recent call last):
  File "/mnt/nfs/projects/ecosystem_biology/local_tools/iRep/iRep", line 73, in <module>
    thresholds, args['no_gc_correction'], args['t'])
  File "/home/users/smartinezarbas/miniconda3/lib/python3.6/site-packages/iRep/iRep.py", line 991, in iRep
    calc_cov_windows(genomes, pairs, mappings, gc_correction, thresholds, threads)
  File "/home/users/smartinezarbas/miniconda3/lib/python3.6/site-packages/iRep/iRep.py", line 838, in calc_cov_windows
    for g, s in pairs]):
  File "/home/users/smartinezarbas/miniconda3/lib/python3.6/multiprocessing/pool.py", line 260, in map
    return self._map_async(func, iterable, mapstar, chunksize).get()
  File "/home/users/smartinezarbas/miniconda3/lib/python3.6/multiprocessing/pool.py", line 608, in get
    raise self._value
  File "/home/users/smartinezarbas/miniconda3/lib/python3.6/multiprocessing/pool.py", line 385, in _handle_tasks
    put(task)
  File "/home/users/smartinezarbas/miniconda3/lib/python3.6/multiprocessing/connection.py", line 206, in send
    self._send_bytes(_ForkingPickler.dumps(obj))
  File "/home/users/smartinezarbas/miniconda3/lib/python3.6/multiprocessing/connection.py", line 393, in _send_bytes
    header = struct.pack("!i", n)
struct.error: 'i' format requires -2147483648 <= number <= 2147483647

The other difference between my initial test run is that I ran it on a passive mode, expecting the whole process to take long.

Have you ever ran it on such a large dataset? Any idea what is the error above?

Best regards, Shaman

[christophertbrown]

Hi Shaman,

I'm not sure what is causing this error, but it looks to me like there may be something wrong with one of the sam files.

When I run iRep on many sam files, I usually run each of them separately. Then, combine the results using iRep_filter.py. This is faster because you can read through multiple sam files at once (I/O on your system usually limiting). I suggest taking this strategy on your large dataset. This will also make it easy to determine if it is one, a few, or all of the sam files that are causing the problem.

Let me know how this goes and if you have any further questions. Hope this helps.

Best,

Chris

On Sep 9, 2017, at 5:45 AM, Shaman notifications@github.com wrote:

Hi Chris,

A colleague of mine managed to install it on our cluster. It also worked for me when I tested in on a small dataset (three sam files, one genome). However, when I tried to run it on a large dataset (53 SAM files and 220 genomes), it seems to halt giving the following:

parsing mapping files

calculating coverage over sliding windows

Traceback (most recent call last): File "/mnt/nfs/projects/ecosystem_biology/local_tools/iRep/iRep", line 73, in thresholds, args['no_gc_correction'], args['t']) File "/home/users/smartinezarbas/miniconda3/lib/python3.6/site-packages/iRep/iRep.py", line 991, in iRep calc_cov_windows(genomes, pairs, mappings, gc_correction, thresholds, threads) File "/home/users/smartinezarbas/miniconda3/lib/python3.6/site-packages/iRep/iRep.py", line 838, in calc_cov_windows for g, s in pairs]): File "/home/users/smartinezarbas/miniconda3/lib/python3.6/multiprocessing/pool.py", line 260, in map return self._map_async(func, iterable, mapstar, chunksize).get() File "/home/users/smartinezarbas/miniconda3/lib/python3.6/multiprocessing/pool.py", line 608, in get raise self._value File "/home/users/smartinezarbas/miniconda3/lib/python3.6/multiprocessing/pool.py", line 385, in _handle_tasks put(task) File "/home/users/smartinezarbas/miniconda3/lib/python3.6/multiprocessing/connection.py", line 206, in send self._send_bytes(_ForkingPickler.dumps(obj)) File "/home/users/smartinezarbas/miniconda3/lib/python3.6/multiprocessing/connection.py", line 393, in _send_bytes header = struct.pack("!i", n) struct.error: 'i' format requires -2147483648 <= number <= 2147483647

The other difference between my initial test run is that I ran it on a passive mode, expecting the whole process to take long.

Have you ever ran it on such a large dataset? Any idea what is the error above?

Best regards, Shaman

— You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[shaman-narayanasamy]

Roger that, Chris. I will give it a try and get back to you.

Cheers, Shaman

[shaman-narayanasamy]

Chris,

Update: Works like a charm! Thanks for the tip :)

Closing the issue.

Cheers, Shaman