Problem running nanocorrect, no output files

[nickloman]

From a discussion I have been having on email with Adam Hargreaves:

Have been trying this command:

python makerange.py Eco6_48hr_just_2D_readsunixfixed.fasta | parallel -v --progress -P8 'python nanocorrect.py Eco6_48hr {} > Eco6_48hr.{}.corrected.fasta'

I wasn't sure of the -P $(NC_PROCESS) syntax, so played about with it a bit, not sure if the above version is correct.

This gave a lot of files as output, lots of lashow..out and poa.input..fa files along with ..corrected.fasta files which all seem to be empty.

Not sure if this is due to the commands I'm giving or if there's some kind of install issue which is preventing the script running fully.

It was, but not to the actual executable. Have amended and then tried again, this time I get this error:

IOError: [Errno 2] No such file or directory: 'clustal-16751.out' python nanocorrect.py Eco6_48hr 16801:16804 > Eco6_48hr.16801:16804.corrected.fasta Traceback (most recent call last): File "nanocorrect.py", line 193, in (seq, n_reads) = run_poa_and_consensus(overlaps, read_idx) File "nanocorrect.py", line 158, in run_poa_and_consensus consensus = clustal2consensus(out_fn) File "nanocorrect.py", line 86, in clustal2consensus alignment = AlignIO.read(fn, "clustal") File "/usr/local/lib/python2.7/dist-packages/Bio/AlignIO/init.py", line 427, in read first = next(iterator) File "/usr/local/lib/python2.7/dist-packages/Bio/AlignIO/init.py", line 350, in parse with as_handle(handle, 'rU') as fp: File "/usr/lib/python2.7/contextlib.py", line 17, in enter return self.gen.next() File "/usr/local/lib/python2.7/dist-packages/Bio/File.py", line 90, in as_handle with open(handleish, mode, **kwargs) as fp: IOError: [Errno 2] No such file or directory: 'clustal-16801.out' local:0/337/100%/0.0s

Some kind of python module missing?

Can you check that 'poa' is on the $PATH ? e.g. if you run poa in a terminal does it run?

It was, but not to the actual executable. Have amended and then tried again, this time I get this error:

IOError: [Errno 2] No such file or directory: 'clustal-16751.out' python nanocorrect.py Eco6_48hr 16801:16804 > Eco6_48hr.16801:16804.corrected.fasta Traceback (most recent call last): File "nanocorrect.py", line 193, in (seq, n_reads) = run_poa_and_consensus(overlaps, read_idx) File "nanocorrect.py", line 158, in run_poa_and_consensus consensus = clustal2consensus(out_fn) File "nanocorrect.py", line 86, in clustal2consensus alignment = AlignIO.read(fn, "clustal") File "/usr/local/lib/python2.7/dist-packages/Bio/AlignIO/init.py", line 427, in read first = next(iterator) File "/usr/local/lib/python2.7/dist-packages/Bio/AlignIO/init.py", line 350, in parse with as_handle(handle, 'rU') as fp: File "/usr/lib/python2.7/contextlib.py", line 17, in enter return self.gen.next() File "/usr/local/lib/python2.7/dist-packages/Bio/File.py", line 90, in as_handle with open(handleish, mode, **kwargs) as fp: IOError: [Errno 2] No such file or directory: 'clustal-16801.out' local:0/337/100%/0.0s

Some kind of python module missing?

[nickloman]

It is really not clear to me what is going on here. What happens when you just run 'poa' from the command line?

[nickloman]

Alternatively I could log in to your server and try it out, or I can make you a virtual machine to run your data on if you prefer, with all the dependencies set up?

[jts]

I agree with @nickloman that poa is probably the issue. I'll modify nanocorrect to make a clear error if this step can't run.

[AdamHargreaves]

Hi Nick, Jared,

If I just run "poa" from Bash, I get the version/help screen.

Again, if I try running the nanocorrect command again I get this error printed

IOError: [Errno 2] No such file or directory: 'clustal-16651.out' python nanocorrect.py Eco6_48hr_2D 16801:16804 > Eco6_48hr_2D.16801:16804.corrected.fasta Traceback (most recent call last): File "nanocorrect.py", line 193, in (seq, n_reads) = run_poa_and_consensus(overlaps, read_idx) File "nanocorrect.py", line 158, in run_poa_and_consensus consensus = clustal2consensus(out_fn) File "nanocorrect.py", line 86, in clustal2consensus alignment = AlignIO.read(fn, "clustal") File "/usr/local/lib/python2.7/dist-packages/Bio/AlignIO/init.py", line 427, in read first = next(iterator) File "/usr/local/lib/python2.7/dist-packages/Bio/AlignIO/init.py", line 350, in parse with as_handle(handle, 'rU') as fp: File "/usr/lib/python2.7/contextlib.py", line 17, in enter return self.gen.next() File "/usr/local/lib/python2.7/dist-packages/Bio/File.py", line 90, in as_handle with open(handleish, mode, **kwargs) as fp: IOError: [Errno 2] No such file or directory: 'clustal-16801.out' ETA: 0s 0left 0.03avg local:0/337/100%/0.0s

Would re-installing poa be worthwhile do you think?

[jts]

Hi Adam,

Is the file poa-blosum80.mat in the same directory as nanocorrect?

[AdamHargreaves]

It is yes

[jts]

Ok, standby I'm making some changes to improve the error reporting. I'll reply here soon.

While I do that though, do you always get this error message or is it particular to a certain range of reads?

[AdamHargreaves]

Give me a minute, I might have solved the issue...

[AdamHargreaves]

OK so it is now running!

Seemed to be the poa-blosum80.mat file you mentioned Jared. The initial directory I had, the text for this file was in red for whatever reason. I re-cloned nanocorrect from GitHub and replaced this file with the newly downloaded version (white text) and it now seems to be running smoothly.

No idea what I had done to mess the file up! Thanks very much both of you for your help!

[jts]

Great, glad its working. I've just made a bunch of changes to give more informative error messages, hopefully this will help troubleshooting in the future.