timeout errors

[tmosbruger]

I am interested in running Clipper on some iCLIP data and I get this message repeatedly when I run the most recent build:

ERROR:root:transcript timed out local variable 'expanded_Nreads' referenced before assignment

Here is the call

clipper -b 11790X1_P1.clean.bam --input_bam=11790X4_P1.clean.bam -s hg19 -o test --processors=10

I also tried to run the last release of clipper but got this message repeatedly:

clipper -b 11790X1_P1.clean.bam -s hg19 -o test --processors=10

ERROR:root:transcript timed out Expected bytes, got unicode

Thanks!

[olgabot]

that sounds like it could be a python3 compatibility issue. What version of python are you using?

On Thu, Sep 17, 2015 at 11:03 AM tmosbruger notifications@github.com wrote:

I am interested in running Clipper on some iCLIP data and I get this message repeatedly when I run the most recent build:

ERROR:root:transcript timed out local variable 'expanded_Nreads' referenced before assignment

Here is the call

clipper -b 11790X1_P1.clean.bam --input_bam=11790X4_P1.clean.bam -s hg19 -o test --processors=10

I also tried to run the last release of clipper but got this message repeatedly:

clipper -b 11790X1_P1.clean.bam -s hg19 -o test --processors=10

ERROR:root:transcript timed out Expected bytes, got unicode

Thanks!

— Reply to this email directly or view it on GitHub https://github.com/YeoLab/clipper/issues/27.

[mlovci]

Pasting our email conversation here, @tmosbruger, please continue here.

@tmosbruger 10:50 AM (18 minutes ago) Hi Michael, I am interested in running Clipper on some iCLIP data and I get t...

---

@mlovci 10:56 AM (13 minutes ago)

to @tmosbruger
How did you install clipper?

Is your bam file sorted and indexed?

@mlovci 10:56 AM (12 minutes ago) to @tmosbruger

if you don't mind posting the issue here, it will help get it resolved quickly.

https://github.com/yeolab/clipper/issues

---

@tmosbruger 10:59 AM (9 minutes ago) to @mlovci

I ended up installing the prerequisites manually, the HTSeq install hung when the install script fetched prerequisites automatically. Essentially, I ran the install script and every time I hit a missing package, I grabbed the most recent one.

Yes, the bam files are sorted and indexed.

Thanks,

---

@mlovci 11:07 AM (1 minute ago) to @tmosbruger

sorry to hear about the trouble with automatic dependency resolution.

If you're using the most recent commit, then we don't expect that error and this should be reported as a bug. But can you check that you have commit 5036b9fbdbaa87acca0392054c24790b2a4a8164 installed?

---

@tmosbruger 11:12 AM (1 minute ago) to @mlovci

I think I pulled the most recent build a few days ago, so I probably don’t have your most recent commit. Let me try again with the auto install.

[tmosbruger]

I am using python2.7

[gpratt]

Ok follow Mikes advice please update to the latest dev version of clipper. pybedtools switched to python3 style and it doesn't play well with some other packages in clipper. Let me know if it works. I'll put out a new version to pipy soon.

Cheers,

Gabe

Gabriel Pratt Bioinformatics Graduate Student, Yeo Lab University of California San Diego

On Thu, Sep 17, 2015 at 11:25 AM, tmosbruger notifications@github.com wrote:

I am using python2.7

— Reply to this email directly or view it on GitHub https://github.com/YeoLab/clipper/issues/27#issuecomment-141176222.

[tmosbruger]

I pulled the most recent version (git clone git://github.com/YeoLab/clipper.git) and ran the install script using a python2.7 installation with manually installed dependencies. It installs, but I get the same error messages mentioned above.

If I try to install to an alternate python2.7, the auto-install still gets hung on the HTSeq installation

[gpratt]

odd, let me check some stuff on my end and get back to you.

Gabriel Pratt Bioinformatics Graduate Student, Yeo Lab University of California San Diego

On Thu, Sep 17, 2015 at 12:01 PM, tmosbruger notifications@github.com wrote:

I pulled the most recent version (git clone git:// github.com/YeoLab/clipper.git) and ran the install script using a python2.7 installation with manually installed dependencies. It installs, but I get the same error messages mentioned above.

If I try to install to an alternate python2.7, the auto-install still gets hung on the HTSeq installation

— Reply to this email directly or view it on GitHub https://github.com/YeoLab/clipper/issues/27#issuecomment-141190227.

[gpratt]

Can you let me know what version pybedtools is?

Also can you run your command with the argument --debug?

Thank you

Gabriel Pratt Bioinformatics Graduate Student, Yeo Lab University of California San Diego

On Thu, Sep 17, 2015 at 12:02 PM, Gabriel Pratt gpratt@ucsd.edu wrote:

odd, let me check some stuff on my end and get back to you.

Gabriel Pratt Bioinformatics Graduate Student, Yeo Lab University of California San Diego

On Thu, Sep 17, 2015 at 12:01 PM, tmosbruger notifications@github.com wrote:

I pulled the most recent version (git clone git:// github.com/YeoLab/clipper.git) and ran the install script using a python2.7 installation with manually installed dependencies. It installs, but I get the same error messages mentioned above.

If I try to install to an alternate python2.7, the auto-install still gets hung on the HTSeq installation

— Reply to this email directly or view it on GitHub https://github.com/YeoLab/clipper/issues/27#issuecomment-141190227.

[tmosbruger]

pybedtools is version 0.7.0

here is the output using --debug

Traceback (most recent call last): File "/home/u0855942/bin/clipper", line 9, in load_entry_point('clipper==0.2.0', 'console_scripts', 'clipper')() File "/home/u0855942/lib/python2.7/site-packages/clipper-0.2.0-py2.7-linux-x86_64.egg/clipper/src/peakfinder.py", line 674, in call_main main(options) File "/home/u0855942/lib/python2.7/site-packages/clipper-0.2.0-py2.7-linux-x86_64.egg/clipper/src/peakfinder.py", line 576, in main jobs.append(func_star(job)) File "/home/u0855942/lib/python2.7/site-packages/clipper-0.2.0-py2.7-linux-x86_64.egg/clipper/src/peakfinder.py", line 146, in func_star return call_peaks(*varables) File "/home/u0855942/lib/python2.7/site-packages/clipper-0.2.0-py2.7-linux-x86_64.egg/clipper/src/call_peak.py", line 1059, in call_peaks peak_dict['sections'][sect]['expanded_Nreads'] = len(expanded_Nreads) UnboundLocalError: local variable 'expanded_Nreads' referenced before assignment

[tmosbruger]

I tried installing on a mac to see it behaved differently. The automatic dependency installation worked great, but I am still getting the 'ERROR:root:transcript timed out local variable 'expanded_Nreads' referenced before assignment' message when I try and run the program.

[enguy]

I'm seeing the same 'ERROR:root:transcript timed out local variable 'expanded_Nreads' referenced before assignment' error message. Looking through the clipper code, it seems to be an issue in call_peak.py, specifically the call_peaks function. The relevant part of call_peak.py (lines 1031-1059) looks like this:

if user_threshold is None:
    if SloP:
        …
        expanded_Nreads = get_reads_in_interval(cur_interval, array_of_reads)
        sect_read_lengths = read_lengths_from_htseq(expanded_Nreads)
        …
    else:
        threshold = gene_threshold
else:
    threshold = user_threshold
…
peak_dict['sections'][sect]['threshold'] = threshold
peak_dict['sections'][sect]['nreads'] = int(Nreads)
peak_dict['sections'][sect]['expanded_Nreads'] = len(expanded_Nreads)

These lines are the only time expanded_Nreads is referenced in any of the clipper files, as far as I can tell. If user_threshold is not None, or Slop is not set, then the expanded_Nreads variable will never be initialized, and we get an error when the program attempts to save the length of the nonexistent expanded_Nreads into peak_dict. So, it seems like a pretty easy fix to pre-initialize expanded_Nreads outside of the if statements. I'm not sure that I can fix this on my own without breaking something else, but hopefully this is helpful.

[gpratt]

Thank you for pointing out the issue! I'll put out a fix later today.

In the mean time try running the code with the --superlocal option. That should give you better peaks anyway.

Gabriel Pratt Bioinformatics Graduate Student, Yeo Lab University of California San Diego

On Tue, Sep 22, 2015 at 10:02 AM, enguy notifications@github.com wrote:

I'm seeing the same 'ERROR:root:transcript timed out local variable 'expanded_Nreads' referenced before assignment' error message. Looking through the clipper code, it seems to be an issue in call_peak.py, specifically the call_peaks function. The relevant part of call_peak.py (lines 1031-1059) looks like this:

if user_threshold is None: if SloP: … expanded_Nreads = get_reads_in_interval(cur_interval, array_of_reads) sect_read_lengths = read_lengths_from_htseq(expanded_Nreads) … else: threshold = gene_threshold else: threshold = user_threshold … peak_dict['sections'][sect]['threshold'] = threshold peak_dict['sections'][sect]['nreads'] = int(Nreads) peak_dict['sections'][sect]['expanded_Nreads'] = len(expanded_Nreads)

These lines are the only time expanded_Nreads is referenced in any of the clipper files, as far as I can tell. If user_threshold is not None, or Slop is not set, then the expanded_Nreads variable will never be initialized, and we get an error when the program attempts to save the length of the nonexistent expanded_Nreads into peak_dict. So, it seems like a pretty easy fix to pre-initialize expanded_Nreads outside of the if statements.

I'm not sure that I can fix this on my own without breaking something else, but hopefully this is helpful.

— Reply to this email directly or view it on GitHub https://github.com/YeoLab/clipper/issues/27#issuecomment-142349187.

[gpratt]

Hmm, I apologize for the delay all,

Still getting around to fixing the issue. I should have mentioned as as stopgap add in the --superlocal option to peak calling. This will improve the quality of your peaks and also allow you to get peaks.

[roeil]

Better late than never: we have installed the latest build/commit and ran it with python 2.7 yet getting the same error as the op:

ERROR:root:transcript timed out Expected bytes, got unicode

without any results (excluding a newly created empty fitted_clusters file).

Any idea if anyone's still around on how to resolve this (mentioned procedures do not seem to bypass the error)?

tia