Incomplete MAPS output

[aperreault]

Hi again!

I have the following errors when running the program on my own data: usage: PROG [-h] [--BINNING_RANGE BINNING_RANGE] DATASET_NAME OUT_DIR MACS2_PATH GF_PATH LONG_PATH SHORT_PATH BIN_SIZE N_CHROMS OUT_FILE_PATH PROG: error: too few arguments first loading parameters file Traceback (most recent call last): File "/home/bryan/MAPS-master/bin/MAPS/MAPS.py", line 225, in main() File "/home/bryan/MAPS-master/bin/MAPS/MAPS.py", line 222, in main init(p) File "/home/bryan/MAPS-master/bin/MAPS/MAPS.py", line 78, in init input_data = pd.read_csv(p.run_file,sep='=',skip_blank_lines=True, comment='#',index_col=0,header=None) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 529, in parser_f return _read(filepath_or_buffer, kwds) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 295, in _read parser = TextFileReader(filepath_or_buffer, kwds) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 612, in init self._make_engine(self.engine) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 747, in _make_engine self._engine = CParserWrapper(self.f, self.options) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 1119, in init self._reader = _parser.TextReader(src, **kwds) File "pandas/parser.pyx", line 353, in pandas.parser.TextReader.cinit (pandas/parser.c:3246) File "pandas/parser.pyx", line 591, in pandas.parser.TextReader._setup_parser_source (pandas/parser.c:6111) IOError: File output_v3/MAPS_output/filename_20190409_102759/maps_filename.maps does not exist second Loading required package: methods Loading required package: stats4 Loading required package: splines [1] "output_v3/MAPS_output/filename_20190409_102759/" [2] "filename.5k"
[3] "5000"
[4] "19"
[5] "None"
[6] "pospoisson"
[1] "filter used (if any):" [1] "None" [1] "loading chromosome chr1 .and" Error in file(file, "rt") : cannot open the connection Calls: read.table -> file Execution halted Error in file(file, "rt") : cannot open the connection Calls: read.table -> file In addition: Warning message: In file(file, "rt") : cannot open file '/output_v3/MAPS_output/filename_20190409_102759/filename.5k.2.peaks': No such file or directory Execution halted third

The feather_output/filename_current directory has 120 files. Is there something in the run_pipeline I've somehow messed up?

Thanks! -Andrea

[armenabnousi]

Hi Andrea,

Can you please copy the entire output, I think there should be one line before usage: PROG [-h] [--BINNING_RANGE BINNING_RANGE] that will give us the exact command that has been called before this error. Also it seems like the preprocessing part is over, and this error happens during interaction calling. can you please check to see if there are chr*.long.intra.bedpe files in the feather_output directory. Thanks!

[aperreault]

Here is the full output (I've redacted some of the full pathing): $ '/home/bryan/MAPS-master/bin/run_pipeline_AP_v2.sh' 2.7.13 |Anaconda custom (64-bit)| (default, Dec 20 2016, 23:09:15) [GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] Tue Apr 9 10:27:59 2019 starting mapping and filtering operation Tue Apr 9 10:27:59 2019 calling bwa for filename_R1.fastq.gz Tue Apr 9 14:03:04 2019 calling bwa forfilename_R2.fastq.gz Tue Apr 9 17:43:16 2019 calling samtools sort for output_v3/feather_output/filename_20190409_102759/tempfiles/filename_R1.fastq.gz.bwa.sam storing in output_v3/feather_output/filename_20190409_102759/tempfiles/filename_R1.fastq.gz.bwa.sam.srtn Tue Apr 9 17:59:46 2019 calling samtools sort for /output_v3/feather_output/filename_20190409_102759/tempfiles/filename_R2.fastq.gz.bwa.sam storing in output_v3/feather_output/filename_20190409_102759/tempfiles/filename_R2.fastq.gz.bwa.sam.srtn Tue Apr 9 18:20:24 2019 merging output_v3/feather_output/filename_20190409_102759/tempfiles/filename_R1.fastq.gz.bwa.sam.srtn and output_v3/feather_output/filename_20190409_102759/tempfiles/filename_R2.fastq.gz.bwa.sam.srtn Tue Apr 9 19:04:19 2019 filtering and pairing reads Tue Apr 9 19:58:38 2019 paired bam file generated. Sorting by coordinates. Tue Apr 9 20:16:05 2019 calling samtools rmdup Tue Apr 9 20:42:11 2019 calling samtools flagstat on mapped file Tue Apr 9 20:44:50 2019 calling samtools flagstat on mapped and duplicate-removed file Tue Apr 9 20:47:17 2019 calling samtools sort for sorting by query names Tue Apr 9 21:07:20 2019 finishing filtering Tue Apr 9 21:07:20 2019 starting the splitting operation NOTE: Too many chromosomes. Generating hic only for ones without an underscore in their name Tue Apr 9 21:58:36 2019 generating long, intra-chromosomal bedpe file(s) no bedpe generated forchr1_GL456210_random. Empty bam file. no bedpe generated forchr1_GL456211_random. Empty bam file. no bedpe generated forchr1_GL456212_random. Empty bam file. no bedpe generated forchr1_GL456213_random. Empty bam file. no bedpe generated forchr1_GL456221_random. Empty bam file. no bedpe generated forchr4_JH584292_random. Empty bam file. no bedpe generated forchr4_GL456350_random. Empty bam file. no bedpe generated forchr4_JH584293_random. Empty bam file. no bedpe generated forchr4_JH584294_random. Empty bam file. no bedpe generated forchr4_JH584295_random. Empty bam file. no bedpe generated forchr5_JH584296_random. Empty bam file. no bedpe generated forchr5_JH584297_random. Empty bam file. no bedpe generated forchr5_JH584298_random. Empty bam file. no bedpe generated forchr5_GL456354_random. Empty bam file. no bedpe generated forchr5_JH584299_random. Empty bam file. no bedpe generated forchr7_GL456219_random. Empty bam file. no bedpe generated forchrY_JH584300_random. Empty bam file. no bedpe generated forchrY_JH584301_random. Empty bam file. no bedpe generated forchrY_JH584302_random. Empty bam file. no bedpe generated forchrY_JH584303_random. Empty bam file. no bedpe generated forchrUn_GL456378. Empty bam file. no bedpe generated forchrUn_GL456392. Empty bam file. no bedpe generated forchrUn_GL456394. Empty bam file. no bedpe generated forchrUn_GL456359. Empty bam file. no bedpe generated forchrUn_GL456372. Empty bam file. no bedpe generated forchrUn_GL456387. Empty bam file. no bedpe generated forchrUn_GL456368. Empty bam file. Tue Apr 9 21:59:30 2019 writing to the combined bedpe file Tue Apr 9 22:00:14 2019 splitting completed filename output_v3/MAPS_output/filename_20190409_102759/ /home/bryan/MAPS-master/MAPS_data_files/mm10/genomic_features/F_GC_M_MboI_5Kb_el.mm10.txt output_v3/feather_output/filename_current/ output_v3/feather_output/filename_current/ 5000 19 output_v3/MAPS_output/filename_20190409_102759/ usage: PROG [-h] [--BINNING_RANGE BINNING_RANGE] DATASET_NAME OUT_DIR MACS2_PATH GF_PATH LONG_PATH SHORT_PATH BIN_SIZE N_CHROMS OUT_FILE_PATH PROG: error: too few arguments first loading parameters file Traceback (most recent call last): File "/home/bryan/MAPS-master/bin/MAPS/MAPS.py", line 225, in main() File "/home/bryan/MAPS-master/bin/MAPS/MAPS.py", line 222, in main init(p) File "/home/bryan/MAPS-master/bin/MAPS/MAPS.py", line 78, in init input_data = pd.read_csv(p.run_file,sep='=',skip_blank_lines=True, comment='#',index_col=0,header=None) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 529, in parser_f return _read(filepath_or_buffer, kwds) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 295, in _read parser = TextFileReader(filepath_or_buffer, kwds) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 612, in init self._make_engine(self.engine) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 747, in _make_engine self._engine = CParserWrapper(self.f, self.options) File "/home/bryan/anaconda2/lib/python2.7/site-packages/pandas/io/parsers.py", line 1119, in init self._reader = _parser.TextReader(src, **kwds) File "pandas/parser.pyx", line 353, in pandas.parser.TextReader.cinit (pandas/parser.c:3246) File "pandas/parser.pyx", line 591, in pandas.parser.TextReader._setup_parser_source (pandas/parser.c:6111) IOError: File output_v3/MAPS_output/filename_20190409_102759/maps_filename.maps does not exist second Loading required package: methods Loading required package: stats4 Loading required package: splines [1] "output_v3/MAPS_output/filename_20190409_102759/" [2] "filename.5k"
[3] "5000"
[4] "19"
[5] "None"
[6] "pospoisson"
[1] "filter used (if any):" [1] "None" [1] "loading chromosome chr1 .and" Error in file(file, "rt") : cannot open the connection Calls: read.table -> file Execution halted Error in file(file, "rt") : cannot open the connection Calls: read.table -> file In addition: Warning message: In file(file, "rt") : cannot open file 'output_v3/MAPS_output/filename_20190409_102759/filename.5k.2.peaks': No such file or directory Execution halted third

And yes, there are chr*.long.intra.bedpe files in the feather_output directory.

[armenabnousi]

Andrea, any chance the macs2_filepath variable in the run_pipeline.sh file is not set? If it is, can you check lines 7 and 145 of the run_pipeline.sh file to make sure there aren't any accidental changes there? (it should be:) macs2_filepath=/blah/blah/blah (your file path) and $python_path $cwd/MAPS/make_maps_runfile.py $dataset_name $maps_output $macs2_filepath $genomic_feat_filepath $long_bedpe_dir $short_bed_dir $bin_size $chr_count $maps_output

[aperreault]

From the run_pipeline.sh it seems as if this path is to the narrowPeaks output of MACS- I have the macs2_filepath blank because I don't have MACS peaks for my HiChIP data. Can MAPS not determine these peaks within the program? Do you need to run MACS before running MAPS?

[armenabnousi]

MAPS requires the ChIP-Seq peaks or ATAC-Seq peaks to run. It is highly recommended that you provide these peaks. If it's not possible, then you can use other software for example hichipper to call the 1D peaks, and then use those 1D peaks as your macs2 file.

[aperreault]

Ah, I thought MAPS also found the peaks. Thanks for the clarification!

-Andrea