search

ijuric / MAPS

18 stars 11 forks source link

AttributeError #5

Closed aperreault closed 5 years ago

aperreault commented 5 years ago

Hi there.

I'm trying to run the test data on my computer to make sure everything is working. Below is the log, with errors bolded. My working directory is the MAPS-master/bin directory and I'm outputting to a different location.

$ run_pipeline_APtest.sh' Fri Apr 5 15:09:57 2019 starting mapping and filtering operation Fri Apr 5 15:09:57 2019 calling bwa for /home/bryan/MAPS-master/examples/test_set1/test_set1_R1.fastq Fri Apr 5 15:10:09 2019 calling bwa for /home/bryan/MAPS-master/examples/test_set1/test_set1_R2.fastq Fri Apr 5 15:10:21 2019 calling samtools sort for /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_20190405_150957/tempfiles/test_set1_R1.fastq.bwa.sam storing in /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_20190405_150957/tempfiles/test_set1_R1.fastq.bwa.sam.srtn Fri Apr 5 15:10:22 2019 calling samtools sort for /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_20190405_150957/tempfiles/test_set1_R2.fastq.bwa.sam storing in /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_20190405_150957/tempfiles/test_set1_R2.fastq.bwa.sam.srtn Fri Apr 5 15:10:23 2019 merging /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_20190405_150957/tempfiles/test_set1_R1.fastq.bwa.sam.srtn and /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_20190405_150957/tempfiles/test_set1_R2.fastq.bwa.sam.srtn Fri Apr 5 15:10:26 2019 filtering and pairing reads Traceback (most recent call last): File "/home/bryan/MAPS-master/bin/feather/feather_pipe", line 116, in main() File "/home/bryan/MAPS-master/bin/feather/feather_pipe", line 49, in main filter_output_filename = filter_main(fastq1, fastq2, bwa_index, mapq, outdir, prefix, threads, to_file = False) File "/home/bryan/MAPS-master/bin/feather/feather_filter_chr.py", line 59, in filter_main filter_pair_reads(combined_bwa_filename, mapq, paired_filename, qc_filename) File "/home/bryan/MAPS-master/bin/feather/feather_filter_chr.py", line 135, in filter_pair_reads paired_reads.append(pair_up(aligned_reads)) File "/home/bryan/MAPS-master/bin/feather/feather_filter_chr.py", line 252, in pair_up r1_cp.next_reference_name = r2_cp.next_reference_name = "=" AttributeError: attribute 'next_reference_name' of 'pysam.calignedsegment.AlignedSegment' objects is not writable sed: can't read /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_20190405_150957/test_set1.feather.qc: No such file or directory test_set1 /home/bryan/MAPS-master/examples/test_set1-output/MAPS_output/test_set1_20190405_150957/ /home/bryan/MAPS-master/examples/test_set1/macs2_peaks_final.replicated.narrowPeak /home/bryan/MAPS-master/MAPS_data_files/mm10/genomic_features/F_GC_M_MboI_5Kb_el.mm10.txt /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_current/ /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_current/ 5000 19 /home/bryan/MAPS-master/examples/test_set1-output/MAPS_output/test_set1_20190405_150957/ first loading parameters file {'BINNING_RANGE': 1000000, 'OUT_DIR': '/home/bryan/MAPS-master/examples/test_set1-output/MAPS_output/test_set1_20190405_150957/', 'SHORT_PATH': '/home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_current/', 'GF_PATH': '/home/bryan/MAPS-master/MAPS_data_files/mm10/genomic_features/F_GC_M_MboI_5Kb_el.mm10.txt', 'LONG_FORMAT': '[DATASET_NAME].[CHROMOSOME].long.intra.bedpe', 'BIN_SIZE': 5000, 'DATASET_NAME': 'test_set1', 'N_CHROMS': 19, 'SHORT_FORMAT': '[DATASET_NAME].[CHROMOSOME].shrt.vip.bed', 'MACS2_PATH': '/home/bryan/MAPS-master/examples/test_set1/macs2_peaks_final.replicated.narrowPeak', 'LONG_PATH': '/home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_current/'} ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19'] loading MACS2 peaks loading metadata file doing chromosome chr1

-- handling MACS2 peaks -- handling short.bed

Traceback (most recent call last): File "/home/bryan/MAPS-master/bin/MAPS/MAPS.py", line 225, in main() File "/home/bryan/MAPS-master/bin/MAPS/MAPS.py", line 222, in main init(p) File "/home/bryan/MAPS-master/bin/MAPS/MAPS.py", line 109, in init ps_short = pd.read_csv(parse_fname(CHR, 'short', params),header=None,sep='\t') File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 529, in parser_f return _read(filepath_or_buffer, kwds) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 295, in _read parser = TextFileReader(filepath_or_buffer, kwds) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 612, in init self._make_engine(self.engine) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 747, in _make_engine self._engine = CParserWrapper(self.f, self.options) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 1119, in init self._reader = _parser.TextReader(src, kwds) File "pandas/parser.pyx", line 353, in pandas.parser.TextReader.cinit (pandas/parser.c:3246) File "pandas/parser.pyx", line 591, in pandas.parser.TextReader._setup_parser_source (pandas/parser.c:6111) IOError: File /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_current/test_set1.chr1.shrt.vip.bed does not exist** second Loading required package: methods Loading required package: stats4 Loading required package: splines [1] "/home/bryan/MAPS-master/examples/test_set1-output/MAPS_output/test_set1_20190405_150957/" [2] "test_set1.5k"
[3] "5000"
[4] "19"
[5] "None"
[6] "pospoisson"
[1] "filter used (if any):" [1] "None" [1] "loading chromosome chr1 .and" Error in file(file, "rt") : cannot open the connection Calls: read.table -> file Execution halted Error in file(file, "rt") : cannot open the connection Calls: read.table -> file In addition: Warning message: In file(file, "rt") : cannot open file '/home/bryan/MAPS-master/examples/test_set1-output/MAPS_output/test_set1_20190405_150957/test_set1.5k.2.peaks': No such file or directory Execution halted third

I appreciate any advice! -Andrea

armenabnousi commented 5 years ago

Hi Andrea,

We've seen that problem before when an older version of pysam was used. Please make sure you have a more recent version of pysam installed.

Generally pysam can be updated by a command like: "pip install pysam --upgrade", if you have admin permissions or are using a locally installed python/pip.

Please let us know if the problem is not solved after upgrading pysam.

Thanks!

On Fri, Apr 5, 2019, 4:56 PM Andrea notifications@github.com wrote:

Hi there.

I'm trying to run the test data on my computer to make sure everything is working. Below is the log, with errors bolded. My working directory is the MAPS-master/bin directory and I'm outputting to a different location.

$ run_pipeline_APtest.sh' Fri Apr 5 15:09:57 2019 starting mapping and filtering operation Fri Apr 5 15:09:57 2019 calling bwa for /home/bryan/MAPS-master/examples/test_set1/test_set1_R1.fastq Fri Apr 5 15:10:09 2019 calling bwa for /home/bryan/MAPS-master/examples/test_set1/test_set1_R2.fastq Fri Apr 5 15:10:21 2019 calling samtools sort for /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_20190405_150957/tempfiles/test_set1_R1.fastq.bwa.sam storing in /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_20190405_150957/tempfiles/test_set1_R1.fastq.bwa.sam.srtn Fri Apr 5 15:10:22 2019 calling samtools sort for /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_20190405_150957/tempfiles/test_set1_R2.fastq.bwa.sam storing in /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_20190405_150957/tempfiles/test_set1_R2.fastq.bwa.sam.srtn Fri Apr 5 15:10:23 2019 merging /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_20190405_150957/tempfiles/test_set1_R1.fastq.bwa.sam.srtn and /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_20190405_150957/tempfiles/test_set1_R2.fastq.bwa.sam.srtn Fri Apr 5 15:10:26 2019 filtering and pairing reads Traceback (most recent call last): File "/home/bryan/MAPS-master/bin/feather/feather_pipe", line 116, in main() File "/home/bryan/MAPS-master/bin/feather/feather_pipe", line 49, in main filter_output_filename = filter_main(fastq1, fastq2, bwa_index, mapq, outdir, prefix, threads, to_file = False) File "/home/bryan/MAPS-master/bin/feather/feather_filter_chr.py", line 59, in filter_main filter_pair_reads(combined_bwa_filename, mapq, paired_filename, qc_filename) File "/home/bryan/MAPS-master/bin/feather/feather_filter_chr.py", line 135, in filter_pair_reads paired_reads.append(pair_up(aligned_reads)) File "/home/bryan/MAPS-master/bin/feather/feather_filter_chr.py", line 252, in pair_up r1_cp.next_reference_name = r2_cp.next_reference_name = "=" AttributeError: attribute 'next_reference_name' of 'pysam.calignedsegment.AlignedSegment' objects is not writable sed: can't read /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_20190405_150957/test_set1.feather.qc: No such file or directory test_set1 /home/bryan/MAPS-master/examples/test_set1-output/MAPS_output/test_set1_20190405_150957/ /home/bryan/MAPS-master/examples/test_set1/macs2_peaks_final.replicated.narrowPeak /home/bryan/MAPS-master/MAPS_data_files/mm10/genomic_features/F_GC_M_MboI_5Kb_el.mm10.txt /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_current/ /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_current/ 5000 19 /home/bryan/MAPS-master/examples/test_set1-output/MAPS_output/test_set1_20190405_150957/ first loading parameters file {'BINNING_RANGE': 1000000, 'OUT_DIR': '/home/bryan/MAPS-master/examples/test_set1-output/MAPS_output/test_set1_20190405_150957/', 'SHORT_PATH': '/home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_current/', 'GF_PATH': '/home/bryan/MAPS-master/MAPS_data_files/mm10/genomic_features/F_GC_M_MboI_5Kb_el.mm10.txt', 'LONG_FORMAT': '[DATASET_NAME].[CHROMOSOME].long.intra.bedpe', 'BIN_SIZE': 5000, 'DATASET_NAME': 'test_set1', 'N_CHROMS': 19, 'SHORT_FORMAT': '[DATASET_NAME].[CHROMOSOME].shrt.vip.bed', 'MACS2_PATH': '/home/bryan/MAPS-master/examples/test_set1/macs2_peaks_final.replicated.narrowPeak', 'LONG_PATH': '/home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_current/'} ['chr1', 'chr2', 'chr3', 'chr4', 'chr5', 'chr6', 'chr7', 'chr8', 'chr9', 'chr10', 'chr11', 'chr12', 'chr13', 'chr14', 'chr15', 'chr16', 'chr17', 'chr18', 'chr19'] loading MACS2 peaks loading metadata file doing chromosome chr1

-- handling MACS2 peaks -- handling short.bed

Traceback (most recent call last): File "/home/bryan/MAPS-master/bin/MAPS/MAPS.py", line 225, in main() File "/home/bryan/MAPS-master/bin/MAPS/MAPS.py", line 222, in main init(p) File "/home/bryan/MAPS-master/bin/MAPS/MAPS.py", line 109, in init ps_short = pd.read_csv(parse_fname(CHR, 'short', params),header=None,sep='\t') File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 529, in parser_f return _read(filepath_or_buffer, kwds) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 295, in _read parser = TextFileReader(filepath_or_buffer, kwds) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 612, in init self._make_engine(self.engine) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 747, in _make_engine self._engine = CParserWrapper(self.f, self.options) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 1119, in init self._reader = _parser.TextReader(src, *kwds) File "pandas/parser.pyx", line 353, in pandas.parser.TextReader.cinit (pandas/parser.c:3246) File "pandas/parser.pyx", line 591, in pandas.parser.TextReader._setup_parser_source (pandas/parser.c:6111) IOError: File /home/bryan/MAPS-master/examples/test_set1-output/feather_output/test_set1_current/test_set1.chr1.shrt.vip.bed does not exist* second Loading required package: methods Loading required package: stats4 Loading required package: splines [1] "/home/bryan/MAPS-master/examples/test_set1-output/MAPS_output/test_set1_20190405_150957/" [2] "test_set1.5k" [3] "5000" [4] "19" [5] "None" [6] "pospoisson" [1] "filter used (if any):" [1] "None" [1] "loading chromosome chr1 .and" Error in file(file, "rt") : cannot open the connection Calls: read.table -> file Execution halted Error in file(file, "rt") : cannot open the connection Calls: read.table -> file In addition: Warning message: In file(file, "rt") : cannot open file '/home/bryan/MAPS-master/examples/test_set1-output/MAPS_output/test_set1_20190405_150957/test_set1.5k.2.peaks': No such file or directory Execution halted third

I appreciate any advice! -Andrea

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/ijuric/MAPS/issues/5, or mute the thread https://github.com/notifications/unsubscribe-auth/AQsPXMDvXLom31i9tkTuULt3GREVQuBJks5vd7hogaJpZM4cftnK .

aperreault commented 5 years ago

Hi! I updated pysam and am still getting errors. Any other advice?

-Andrea

armenabnousi commented 5 years ago

Could you please try running <python> -c 'import pysam; print(pysam.__version__)' and let me know the result. Make sure you replace the <python> to point to the same file that you have on the second line of the run_pipeline.sh file. (For example, in the default run_pipeline.sh, line 2 is: "python_path=/home/abnousa/software/python3.6.5/bin/python" so I run: /home/abnousa/software/python3.6.5/bin/python -c 'import pysam; print(pysam.__version__)' which outputs 0.14.1 (that's my pysam version)

aperreault commented 5 years ago

$ /home/bryan/anaconda2/bin/python2.7 -c 'import pysam; print(pysam.version)' 0.15.2

armenabnousi commented 5 years ago

and the error you are getting is still the same: attribute 'next_reference_name' of 'pysam.calignedsegment.AlignedSegment' objects is not writable

I updated my pysam to 0.15.2 but I can't reproduce that error.

I also noticed you are using python2.7 and I'm confused how you get to this point since python2 would give you errors before reaching here. Could you please add this two lines on top of the "feather/feather_pipe" file,

import sys
print(sys.version)

and let me know what prints at the beginning when you try running MAPS. This will ensure that your MAPS is really running using "/home/bryan/anaconda2/bin/python2.7"

Thanks!

aperreault commented 5 years ago

That worked! I have output in both the feather and MAPS output directories. There must've been a lag in my saved scripts. The print line shows the program is using python 2.7.13. Is this a problem? Are my output files valid even if I'm using python2?

Is there an example of what the output of the test data should look like on the GitHub repo? Right now, my MAPS_output/test_set1_current directory has a ton of files in it and I'm not sure if they're correct.

Thanks!

aperreault commented 5 years ago

Never mind, I found the examples .peaks.bedpe file I can compare to.

Another question though- which file can be uploaded to WashU Epigenome browser for visualization of loops?

armenabnousi commented 5 years ago

Glad it worked. The bedpe file can be uploaded to UCSD browser if I remember right. For WashU you will need to convert bedpe file to tabix. I just added my script for this to github here. First you will need to install tabix though. This script will give you two files, one .tabix and another .tabix.gz, both files need to be uploaded to the WashU browser for visualization.

aperreault commented 5 years ago

Thank you so much for your quick responses and helpful comments! I really appreciate it.

Have a great day! -Andrea

armenabnousi commented 5 years ago

glad could be of help! :)