search

aertslab / scenicplus

SCENIC+ is a python package to build gene regulatory networks (GRNs) using combined or separate single-cell gene expression (scRNA-seq) and single-cell chromatin accessibility (scATAC-seq) data.
Other
186 stars 29 forks source link

export_pseudobulk() does not work #297

Closed wgao688 closed 9 months ago

wgao688 commented 9 months ago

Describe the bug

I am running the 10X pbmc tutorial for SCENIC+ and there is an error at pseudobulk_peak_calling. First, it does not accept optional arguments that it should. Then, after removing these optional arguments, the function is not able to find a file that is present.

To Reproduce

I run the https://scenicplus.readthedocs.io/en/latest/pbmc_multiome_tutorial.html from here exactly the same way, up to this code block.

from pycisTopic.pseudobulk_peak_calling import export_pseudobulk

bw_paths, bed_paths = export_pseudobulk(input_data = cell_data, variable = 'celltype', sample_id_col = 'sample_id', chromsizes = chromsizes, bed_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bed_files/'), bigwig_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bw_files/'), path_to_fragments = fragments_dict, n_cpu = 8, normalize_bigwig = True, remove_duplicates = True, _temp_dir = os.path.join(tmp_dir, 'ray_spill'), split_pattern = '-')

Error output

TypeError Traceback (most recent call last) Cell In[23], line 2 1 from pycisTopic.pseudobulk_peak_calling import export_pseudobulk ----> 2 bw_paths, bed_paths = export_pseudobulk(input_data = cell_data, 3 variable = 'celltype', # variable by which to generate pseubulk profiles, in this case we want pseudobulks per celltype 4 sample_id_col = 'sample_id', 5 chromsizes = chromsizes, 6 bed_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bed_files/'), # specify where pseudobulk_bed_files should be stored 7 bigwig_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bw_files/'),# specify where pseudobulk_bw_files should be stored 8 path_to_fragments = fragments_dict, # location of fragment fiels 9 n_cpu = 8, # specify the number of cores to use, we use ray for multi processing 10 normalize_bigwig = True, 11 remove_duplicates = True, 12 _temp_dir = os.path.join(tmp_dir, 'ray_spill'), 13 split_pattern = '-')

TypeError: export_pseudobulk() got an unexpected keyword argument 'remove_duplicates'

If I remove the "remove_duplicates" argument, then I get this error:

export_pseudobulk() got an unexpected keyword argument '_temp_dir'

If I remove the _temp_dir argument, I get

Could not open file pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz

Expected behavior

I expect that I should be able to generate cell-type specific pseudobulked fragment files.

Version (please complete the following information):

Thanks for your help!

Tu4n-ph4m commented 9 months ago

I also got the same error running the pbmc tutorial!

Aya-Balbaa commented 9 months ago

I am getting the same error as well

laki-spk commented 9 months ago

I am getting the same error. Please help.

MVemon commented 9 months ago

After digging around, it appears that the pseudobulk_peak_calling.py file from the default download version of pycisTopic (v1.0a) is outdated and is missing a couple of the variables found in the tutorial for function export_pseudobulk(). You can compare with this file from their github.

I managed to "fix" it by downloading a version v1.0.2 of pycisTopic, but then there was a plethora of other issues that followed. Hopefully this will be patched up officially soon.

SeppeDeWinter commented 9 months ago

Hi @wgao688, @Tu4n-ph4m, @Aya-Balbaa, @laki-spk and @marck1198

This parameter has been removed as off this PR https://github.com/aertslab/pycisTopic/pull/105. This part of the code has been optimised and it now always removes duplicates.

You can just run the same code without this parameter.

All the best,

Seppe

Aya-Balbaa commented 9 months ago

@SeppeDeWinter Thank you for your reply I tried without remove_duplicates and got this error export_pseudobulk() got an unexpected keyword argument '_temp_dir'

Tu4n-ph4m commented 9 months ago

@Aya-Balbaa I believe the equivalent parameter in the updated file is "temp_dir" only ( without the underscore). Unfortunately, I still got another error though: Could not open file pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz

Aya-Balbaa commented 9 months ago

@Tu4n-ph4m I tried without '_' and got this error: Please, provide a path to fragments for each sample in your cell metadata! I added the path to the meta data, and it's running. I am having an error , but it seems to be related to mismatch in barcodes , '-1' is added

Tu4n-ph4m commented 9 months ago

@Aya-Balbaa unfortunately, it didn't work for me either. I still have this error: PanicException: Could not open file pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz

wgao688 commented 9 months ago

@Tu4n-ph4m @SeppeDeWinter I am also still getting the same error PanicException: Could not open file pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz

After running the command after not supply the the tmp_dir and remove_duplicates options 

from pycisTopic.pseudobulk_peak_calling import export_pseudobulk
bw_paths, bed_paths = export_pseudobulk(input_data = cell_data,
                 variable = 'celltype',                                                                     # variable by which to generate pseubulk profiles, in this case we want pseudobulks per celltype
                 sample_id_col = 'sample_id',
                 chromsizes = chromsizes,
                 bed_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bed_files/'),  # specify where pseudobulk_bed_files should be stored
                 bigwig_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bw_files/'),# specify where pseudobulk_bw_files should be stored
                 path_to_fragments = fragments_dict,                                                        # location of fragment fiels
                 n_cpu = 8,                                                                                 # specify the number of cores to use, we use ray for multi processing
                 normalize_bigwig = True,
                 split_pattern = '-')`
SeppeDeWinter commented 9 months ago

Hi @Aya-Balbaa, @Tu4n-ph4m and @wgao688

Indeed _temp_dir changed to temp_dir, I should have mentioned that as well.

@Tu4n-ph4m and @wgao688

Can you show 


ls pbmc_tutorial/data/

and


zcat pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz | head

Are you sure that you are specifying the correct paths in your fragment dict?

All the best,

Seppe

laki-spk commented 9 months ago

Hi @SeppeDeWinter

I still cannot resolve that error.

This is what I am getting when I typed in your above commands;

(base) [c___@compute-10-10 Scenic]$ ls pbmc_tutorial/data/
pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz
pbmc_granulocyte_sorted_3k_filtered_feature_bc_matrix.h5
(base) [c___@compute-10-10 Scenic]$ zcat pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz | head
# id=pbmc_granulocyte_sorted_3k
# description=PBMC from a healthy donor - granulocytes removed through cell sorting (3k)
#
# pipeline_name=cellranger-arc
# pipeline_version=cellranger-arc-2.0.0
#
# reference_path=/mnt/scratch2/cellranger-arc-2.0.0/refdata-cellranger-arc-GRCh38-2020-A-2.0.0
# reference_fasta_hash=b6f131840f9f337e7b858c3d1e89d7ce0321b243
# reference_gtf_hash=3b4c36ca3bade222a5b53394e8c07a18db7ebb11
# reference_version=2020-A
(base) [ch____@compute-10-10 Scenic]$
SeppeDeWinter commented 9 months ago

Hi @laki-spk

You are missing the index for pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz, this is now required.

You can generate this using


tabix -p bed pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz

All the best.

Seppe

laki-spk commented 9 months ago

Hi @SeppeDeWinter

I am still getting the same error as PanicException: Could not open file pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz

I do have index file also in my data directory. 

(base) [c___@compute-10-7 Scenic]$ ls pbmc_tutorial/data/
pbmc_granulocyte_sorted_10k_atac_fragments.tsv.gz.tbi
pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz
pbmc_granulocyte_sorted_3k_filtered_feature_bc_matrix.h5
(base) [c____@compute-10-7 Scenic]$ zcat pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz | head
# id=pbmc_granulocyte_sorted_3k
# description=PBMC from a healthy donor - granulocytes removed through cell sorting (3k)
#
# pipeline_name=cellranger-arc
# pipeline_version=cellranger-arc-2.0.0
#
# reference_path=/mnt/scratch2/cellranger-arc-2.0.0/refdata-cellranger-arc-GRCh38-2020-A-2.0.0
# reference_fasta_hash=b6f131840f9f337e7b858c3d1e89d7ce0321b243
# reference_gtf_hash=3b4c36ca3bade222a5b53394e8c07a18db7ebb11
# reference_version=2020-A
(base) [c___@compute-10-7 Scenic]$ ^C
(base) [c___@compute-10-7 Scenic]$
MVemon commented 9 months ago

Hello @laki-spk and @SeppeDeWinter,

I can confirm that I am also experiencing the same issue with opening the fragment file. I tested using a different data set from the pycisTopic human cerebellum single-cell multiome data tutorial and experienced the same issue.

Also tried directing it to the file using os.path.abspath() but got the same error.

Thanks for the help @SeppeDeWinter .

SeppeDeWinter commented 9 months ago

Hmm strange.. could you try unzipping the file and rezipping it using bgzip?

Not sure what's going on here. If that does not work I will try to load it myself tomorrow. Just to make sure, it's the fragments from the tutorial?

I hope to have this resolved soon.

All the best,

Seppe

MVemon commented 9 months ago

Hello,

The fragment file was downloaded from the tutorial with the line: !wget -O pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz https://cf.10xgenomics.com/samples/cell-arc/2.0.0/pbmc_granulocyte_sorted_3k/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz. I re-downloaded it to be safe and still got the same error.

I uncompressed and compressed the file with bgzip and both the file types (.tsv) and (.gz) weren't able to be opened.

Lastly, I tried using my own 10x fragment data and still got errors.

Thanks and looking forward to hearing back @SeppeDeWinter.

To be thorough, I'll add the error message below alongside the packages versions:

pycisTopic : Version: 1.0a0 ray: Version: 2.9.2 scanpy: Version: 1.9.8 pyranges: Version: 0.0.127 requests: Version: 2.31.0 pandas: Version: 1.5.0 scenicplus: Version: 1.0.1.dev4+ge4bdd9f

2024-02-14 12:01:17,443 cisTopic     INFO     Splitting fragments by cell type.

[E::idx_find_and_load] Could not retrieve index file for 'pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz'
thread '<unnamed>' panicked at src/split_fragments.rs:93:29:
Could not open file pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---------------------------------------------------------------------------
_RemoteTraceback                          Traceback (most recent call last)
_RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/mvemon/anaconda3/envs/scenicplus/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 463, in _process_worker
    r = call_item()
  File "/home/mvemon/anaconda3/envs/scenicplus/lib/python3.8/site-packages/joblib/externals/loky/process_executor.py", line 291, in __call__
    return self.fn(*self.args, **self.kwargs)
  File "/home/mvemon/anaconda3/envs/scenicplus/lib/python3.8/site-packages/joblib/parallel.py", line 589, in __call__
    return [func(*args, **kwargs)
  File "/home/mvemon/anaconda3/envs/scenicplus/lib/python3.8/site-packages/joblib/parallel.py", line 589, in <listcomp>
    return [func(*args, **kwargs)
pyo3_runtime.PanicException: Could not open file pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz
"""

The above exception was the direct cause of the following exception:

PanicException                            Traceback (most recent call last)
Cell In[24], line 2
      1 from pycisTopic.pseudobulk_peak_calling import export_pseudobulk
----> 2 bw_paths, bed_paths = export_pseudobulk(input_data = cell_data,
      3                  variable = 'celltype',                                                                     # variable by which to generate pseubulk profiles, in this case we want pseudobulks per celltype
      4                  sample_id_col = 'sample_id',
      5                  chromsizes = chromsizes,
      6                  bed_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bed_files/'),  # specify where pseudobulk_bed_files should be stored
      7                  bigwig_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bw_files/'),# specify where pseudobulk_bw_files should be stored
      8                  path_to_fragments = fragments_dict,                                                        # location of fragment fiels
      9                  n_cpu = 8,                                                                                 # specify the number of cores to use, we use ray for multi processing
     10                  normalize_bigwig = True,
     11                  temp_dir = '/tmp/ray_spill',
     12                  split_pattern = '-')

File ~/pycisTopic/python/pycisTopic/pseudobulk_peak_calling.py:156, in export_pseudobulk(input_data, variable, chromsizes, bed_path, bigwig_path, path_to_fragments, sample_id_col, n_cpu, normalize_bigwig, split_pattern, temp_dir)
    153 # For each sample, get fragments for each cell type
    155 log.info("Splitting fragments by cell type.")
--> 156 split_fragment_files_by_cell_type(
    157     sample_to_fragment_file = path_to_fragments,
    158     path_to_temp_folder = temp_dir,
    159     path_to_output_folder = bed_path,
    160     sample_to_cell_type_to_cell_barcodes = sample_to_cell_type_to_barcodes,
    161     chromsizes = chromsizes_dict,
    162     n_cpu = n_cpu,
    163     verbose = False,
    164     clear_temp_folder = True
    165 )
    167 bed_paths = {}
    168 for cell_type in cell_data[variable].unique():

File ~/anaconda3/envs/scenicplus/lib/python3.8/site-packages/scatac_fragment_tools/library/split/split_fragments_by_cell_type.py:72, in split_fragment_files_by_cell_type(sample_to_fragment_file, path_to_temp_folder, path_to_output_folder, sample_to_cell_type_to_cell_barcodes, chromsizes, n_cpu, verbose, clear_temp_folder)
     70 if verbose:
     71     print("Splitting fragments ...")
---> 72 joblib.Parallel(n_jobs=n_cpu)(
     73     joblib.delayed(_rust_scatac_fragment_tools.split_fragments_by_cell_barcode)
     74         (
     75             path_to_fragments = sample_to_fragment_file[sample],
     76             path_to_output_folder = os.path.join(path_to_temp_folder, sample),
     77             cell_type_to_cell_barcodes = sample_to_cell_type_to_cell_barcodes[sample],
     78             chromsizes = chromsizes,
     79             verbose = verbose
     80         )
     81         for sample in sample_to_cell_type_to_cell_barcodes
     82 )
     84 # Check wether all files were create successfully
     85 # and create a dictionary mapping cell types to fragment files
     86 cell_type_to_fragment_files: Dict[str, List[str]] = {}

File ~/anaconda3/envs/scenicplus/lib/python3.8/site-packages/joblib/parallel.py:1952, in Parallel.__call__(self, iterable)
   1946 # The first item from the output is blank, but it makes the interpreter
   1947 # progress until it enters the Try/Except block of the generator and
   1948 # reach the first `yield` statement. This starts the aynchronous
   1949 # dispatch of the tasks to the workers.
   1950 next(output)
-> 1952 return output if self.return_generator else list(output)

File ~/anaconda3/envs/scenicplus/lib/python3.8/site-packages/joblib/parallel.py:1595, in Parallel._get_outputs(self, iterator, pre_dispatch)
   1592     yield
   1594     with self._backend.retrieval_context():
-> 1595         yield from self._retrieve()
   1597 except GeneratorExit:
   1598     # The generator has been garbage collected before being fully
   1599     # consumed. This aborts the remaining tasks if possible and warn
   1600     # the user if necessary.
   1601     self._exception = True

File ~/anaconda3/envs/scenicplus/lib/python3.8/site-packages/joblib/parallel.py:1699, in Parallel._retrieve(self)
   1692 while self._wait_retrieval():
   1693 
   1694     # If the callback thread of a worker has signaled that its task
   1695     # triggered an exception, or if the retrieval loop has raised an
   1696     # exception (e.g. `GeneratorExit`), exit the loop and surface the
   1697     # worker traceback.
   1698     if self._aborting:
-> 1699         self._raise_error_fast()
   1700         break
   1702     # If the next job is not ready for retrieval yet, we just wait for
   1703     # async callbacks to progress.

File ~/anaconda3/envs/scenicplus/lib/python3.8/site-packages/joblib/parallel.py:1734, in Parallel._raise_error_fast(self)
   1730 # If this error job exists, immediatly raise the error by
   1731 # calling get_result. This job might not exists if abort has been
   1732 # called directly or if the generator is gc'ed.
   1733 if error_job is not None:
-> 1734     error_job.get_result(self.timeout)

File ~/anaconda3/envs/scenicplus/lib/python3.8/site-packages/joblib/parallel.py:736, in BatchCompletionCallBack.get_result(self, timeout)
    730 backend = self.parallel._backend
    732 if backend.supports_retrieve_callback:
    733     # We assume that the result has already been retrieved by the
    734     # callback thread, and is stored internally. It's just waiting to
    735     # be returned.
--> 736     return self._return_or_raise()
    738 # For other backends, the main thread needs to run the retrieval step.
    739 try:

File ~/anaconda3/envs/scenicplus/lib/python3.8/site-packages/joblib/parallel.py:754, in BatchCompletionCallBack._return_or_raise(self)
    752 try:
    753     if self.status == TASK_ERROR:
--> 754         raise self._result
    755     return self._result
    756 finally:

PanicException: Could not open file pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz
elhaam commented 9 months ago

Hi Sepp, thanks for the detailed explanations. Above suggestion worked for me and I closed a related issue since it is solved: https://github.com/aertslab/scenicplus/issues/299

Now, the output here is strange and does not match the tutorial. Do you have any suggestions? 

from pycisTopic.pseudobulk_peak_calling import export_pseudobulk
bw_paths, bed_paths = export_pseudobulk(input_data = cell_data,
                 variable = 'celltype',                                                                     # variable by which to generate pseubulk profiles, in this case we want pseudobulks per celltype
                 sample_id_col = 'sample_id',
                 chromsizes = chromsizes,
                 bed_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bed_files/'),  # specify where pseudobulk_bed_files should be stored
                 bigwig_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bw_files/'),# specify where pseudobulk_bw_files should be stored
                 path_to_fragments = fragments_dict,                                                        # location of fragment fiels
                 n_cpu = 8,                                                                                 # specify the number of cores to use, we use ray for multi processing
                 normalize_bigwig = True,
#                  remove_duplicates = True,
                 temp_dir = os.path.join(tmp_dir, 'ray_spill'),
                 split_pattern = '-')
Click here to see my output:

2024-02-14 16:04:42,227 cisTopic INFO Splitting fragments by cell type. 2024-02-14 16:05:16,387 cisTopic INFO generating bigwig files Add chromosome sizes to bigWig header Number of fragments: 986156 Split fragments df by chromosome Calculate depth per chromosome: - GL000009.2 ... - GL000194.1 ... - GL000195.1 ... - GL000205.2 ... - GL000219.1 ... - KI270711.1 ... - KI270713.1 ... - KI270721.1 ... - KI270726.1 ... - KI270727.1 ... - chr1 ... - chr10 ... - chr11 ... - chr12 ... - chr13 ... - chr14 ... - chr15 ... - chr16 ... - chr17 ... - chr18 ... - chr19 ... - chr2 ... - chr20 ... - chr21 ... - chr22 ... - chr3 ... - chr4 ... - chr5 ... - chr6 ... - chr7 ... - chr8 ... - chr9 ... - chrX ... - chrY ... Compact depth array per chromosome (make ranges for consecutive the same values and remove zeros): - Compact GL000008.2 ... - Compact GL000009.2 ... - Write GL000009.2 to bigWig ... - Compact GL000194.1 ... - Write GL000194.1 to bigWig ... - Compact GL000195.1 ... - Write GL000195.1 to bigWig ... - Compact GL000205.2 ... - Write GL000205.2 to bigWig ... - Compact GL000208.1 ... - Compact GL000209.2 ... - Compact GL000213.1 ... - Compact GL000214.1 ... - Compact GL000216.2 ... - Compact GL000218.1 ... - Compact GL000219.1 ... - Write GL000219.1 to bigWig ... - Compact GL000220.1 ... - Compact GL000221.1 ... - Compact GL000224.1 ... - Compact GL000225.1 ... - Compact GL000226.1 ... - Compact GL000250.2 ... - Compact GL000251.2 ... - Compact GL000252.2 ... - Compact GL000253.2 ... - Compact GL000254.2 ... - Compact GL000255.2 ... - Compact GL000256.2 ... - Compact GL000257.2 ... - Compact GL000258.2 ... - Compact GL339449.2 ... - Compact GL383518.1 ... - Compact GL383519.1 ... - Compact GL383520.2 ... - Compact GL383521.1 ... - Compact GL383522.1 ... - Compact GL383526.1 ... - Compact GL383527.1 ... - Compact GL383528.1 ... - Compact GL383530.1 ... - Compact GL383531.1 ... - Compact GL383532.1 ... - Compact GL383533.1 ... - Compact GL383534.2 ... - Compact GL383539.1 ... - Compact GL383540.1 ... - Compact GL383541.1 ... - Compact GL383542.1 ... - Compact GL383545.1 ... - Compact GL383546.1 ... - Compact GL383547.1 ... - Compact GL383549.1 ... - Compact GL383550.2 ... - Compact GL383551.1 ... - Compact GL383552.1 ... - Compact GL383553.2 ... - Compact GL383554.1 ... - Compact GL383555.2 ... - Compact GL383556.1 ... - Compact GL383557.1 ... - Compact GL383563.3 ... - Compact GL383564.2 ... - Compact GL383565.1 ... - Compact GL383566.1 ... - Compact GL383567.1 ... - Compact GL383568.1 ... - Compact GL383569.1 ... - Compact GL383570.1 ... - Compact GL383571.1 ... - Compact GL383572.1 ... - Compact GL383573.1 ... - Compact GL383574.1 ... - Compact GL383575.2 ... - Compact GL383576.1 ... - Compact GL383577.2 ... - Compact GL383578.2 ... - Compact GL383579.2 ... - Compact GL383580.2 ... - Compact GL383581.2 ... - Compact GL383582.2 ... - Compact GL383583.2 ... - Compact GL582966.2 ... - Compact GL877875.1 ... - Compact GL877876.1 ... - Compact GL949742.1 ... - Compact GL949746.1 ... - Compact GL949747.2 ... - Compact GL949748.2 ... - Compact GL949749.2 ... - Compact GL949750.2 ... - Compact GL949751.2 ... - Compact GL949752.1 ... - Compact GL949753.2 ... - Compact JH159136.1 ... - Compact JH159137.1 ... - Compact JH159146.1 ... - Compact JH159147.1 ... - Compact JH159148.1 ... - Compact JH636055.2 ... - Compact KB021644.2 ... - Compact KB663609.1 ... - Compact KI270302.1 ... - Compact KI270303.1 ... - Compact KI270304.1 ... - Compact KI270305.1 ... - Compact KI270310.1 ... - Compact KI270311.1 ... - Compact KI270312.1 ... - Compact KI270315.1 ... - Compact KI270316.1 ... - Compact KI270317.1 ... - Compact KI270320.1 ... - Compact KI270322.1 ... - Compact KI270329.1 ... - Compact KI270330.1 ... - Compact KI270333.1 ... - Compact KI270334.1 ... - Compact KI270335.1 ... - Compact KI270336.1 ... - Compact KI270337.1 ... - Compact KI270338.1 ... - Compact KI270340.1 ... - Compact KI270362.1 ... - Compact KI270363.1 ... - Compact KI270364.1 ... - Compact KI270366.1 ... - Compact KI270371.1 ... - Compact KI270372.1 ... - Compact KI270373.1 ... - Compact KI270374.1 ... - Compact KI270375.1 ... - Compact KI270376.1 ... - Compact KI270378.1 ... - Compact KI270379.1 ... - Compact KI270381.1 ... - Compact KI270382.1 ... - Compact KI270383.1 ... - Compact KI270384.1 ... - Compact KI270385.1 ... - Compact KI270386.1 ... - Compact KI270387.1 ... - Compact KI270388.1 ... - Compact KI270389.1 ... - Compact KI270390.1 ... - Compact KI270391.1 ... - Compact KI270392.1 ... - Compact KI270393.1 ... - Compact KI270394.1 ... - Compact KI270395.1 ... - Compact KI270396.1 ... - Compact KI270411.1 ... - Compact KI270412.1 ... - Compact KI270414.1 ... - Compact KI270417.1 ... - Compact KI270418.1 ... - Compact KI270419.1 ... - Compact KI270420.1 ... - Compact KI270422.1 ... - Compact KI270423.1 ... - Compact KI270424.1 ... - Compact KI270425.1 ... - Compact KI270429.1 ... - Compact KI270435.1 ... - Compact KI270438.1 ... - Compact KI270442.1 ... - Compact KI270448.1 ... - Compact KI270465.1 ... - Compact KI270466.1 ... - Compact KI270467.1 ... - Compact KI270468.1 ... - Compact KI270507.1 ... - Compact KI270508.1 ... - Compact KI270509.1 ... - Compact KI270510.1 ... - Compact KI270511.1 ... - Compact KI270512.1 ... - Compact KI270515.1 ... - Compact KI270516.1 ... - Compact KI270517.1 ... - Compact KI270518.1 ... - Compact KI270519.1 ... - Compact KI270521.1 ... - Compact KI270522.1 ... - Compact KI270528.1 ... - Compact KI270529.1 ... - Compact KI270530.1 ... - Compact KI270538.1 ... - Compact KI270539.1 ... - Compact KI270544.1 ... - Compact KI270548.1 ... - Compact KI270579.1 ... - Compact KI270580.1 ... - Compact KI270581.1 ... - Compact KI270582.1 ... - Compact KI270583.1 ... - Compact KI270584.1 ... - Compact KI270587.1 ... - Compact KI270588.1 ... - Compact KI270589.1 ... - Compact KI270590.1 ... - Compact KI270591.1 ... - Compact KI270593.1 ... - Compact KI270706.1 ... - Compact KI270707.1 ... - Compact KI270708.1 ... - Compact KI270709.1 ... - Compact KI270710.1 ... - Compact KI270711.1 ... - Write KI270711.1 to bigWig ... - Compact KI270712.1 ... - Compact KI270713.1 ... - Write KI270713.1 to bigWig ... - Compact KI270714.1 ... - Compact KI270715.1 ... - Compact KI270716.1 ... - Compact KI270717.1 ... - Compact KI270718.1 ... - Compact KI270719.1 ... - Compact KI270720.1 ... - Compact KI270721.1 ... - Write KI270721.1 to bigWig ... - Compact KI270722.1 ... - Compact KI270723.1 ... - Compact KI270724.1 ... - Compact KI270725.1 ... - Compact KI270726.1 ... - Write KI270726.1 to bigWig ... - Compact KI270727.1 ... - Write KI270727.1 to bigWig ... - Compact KI270728.1 ... - Compact KI270729.1 ... - Compact KI270730.1 ... - Compact KI270731.1 ... - Compact KI270732.1 ... - Compact KI270733.1 ... - Compact KI270734.1 ... - Compact KI270735.1 ... - Compact KI270736.1 ... - Compact KI270737.1 ... - Compact KI270738.1 ... - Compact KI270739.1 ... - Compact KI270740.1 ... - Compact KI270741.1 ... - Compact KI270742.1 ... - Compact KI270743.1 ... - Compact KI270744.1 ... - Compact KI270745.1 ... - Compact KI270746.1 ... - Compact KI270747.1 ... - Compact KI270748.1 ... - Compact KI270749.1 ... - Compact KI270750.1 ... - Compact KI270751.1 ... - Compact KI270752.1 ... - Compact KI270753.1 ... - Compact KI270754.1 ... - Compact KI270755.1 ... - Compact KI270756.1 ... - Compact KI270757.1 ... - Compact KI270758.1 ... - Compact KI270759.1 ... - Compact KI270760.1 ... - Compact KI270761.1 ... - Compact KI270762.1 ... - Compact KI270763.1 ... - Compact KI270764.1 ... - Compact KI270765.1 ... - Compact KI270766.1 ... - Compact KI270767.1 ... - Compact KI270768.1 ... - Compact KI270769.1 ... Add chromosome sizes to bigWig header Number of fragments: 540829 Split fragments df by chromosome Calculate depth per chromosome: - GL000194.1 ... - GL000195.1 ... - GL000205.2 ... - GL000219.1 ... - KI270711.1 ... - KI270713.1 ... - KI270721.1 ... - KI270726.1 ... - KI270734.1 ... - chr1 ... - chr10 ... - chr11 ... - chr12 ... - chr13 ... - chr14 ... - chr15 ... - chr16 ... - chr17 ... - chr18 ... - chr19 ... - chr2 ... - chr20 ... - chr21 ... - chr22 ... - chr3 ... - chr4 ... - chr5 ... - chr6 ... - chr7 ... - chr8 ... - chr9 ... - chrX ... - chrY ... Compact depth array per chromosome (make ranges for consecutive the same values and remove zeros): - Compact GL000008.2 ... - Compact GL000009.2 ... - Compact GL000194.1 ... - Write GL000194.1 to bigWig ... - Compact GL000195.1 ... - Write GL000195.1 to bigWig ... - Compact GL000205.2 ... - Write GL000205.2 to bigWig ... - Compact GL000208.1 ... - Compact GL000209.2 ... - Compact GL000213.1 ... - Compact GL000214.1 ... - Compact GL000216.2 ... - Compact GL000218.1 ... - Compact GL000219.1 ... - Write GL000219.1 to bigWig ... - Compact GL000220.1 ... - Compact GL000221.1 ... - Compact GL000224.1 ... - Compact GL000225.1 ... - Compact GL000226.1 ... - Compact GL000250.2 ... - Compact GL000251.2 ... - Compact GL000252.2 ... - Compact GL000253.2 ... - Compact GL000254.2 ... - Compact GL000255.2 ... - Compact GL000256.2 ... - Compact GL000257.2 ... - Compact GL000258.2 ... - Compact GL339449.2 ... - Compact GL383518.1 ... - Compact GL383519.1 ... - Compact GL383520.2 ... - Compact GL383521.1 ... - Compact GL383522.1 ... - Compact GL383526.1 ... - Compact GL383527.1 ... - Compact GL383528.1 ... - Compact GL383530.1 ... - Compact GL383531.1 ... - Compact GL383532.1 ... - Compact GL383533.1 ... - Compact GL383534.2 ... - Compact GL383539.1 ... - Compact GL383540.1 ... - Compact GL383541.1 ... - Compact GL383542.1 ... - Compact GL383545.1 ... - Compact GL383546.1 ... - Compact GL383547.1 ... - Compact GL383549.1 ... - Compact GL383550.2 ... - Compact GL383551.1 ... - Compact GL383552.1 ... - Compact GL383553.2 ... - Compact GL383554.1 ... - Compact GL383555.2 ... - Compact GL383556.1 ... - Compact GL383557.1 ... - Compact GL383563.3 ... - Compact GL383564.2 ... - Compact GL383565.1 ... - Compact GL383566.1 ... - Compact GL383567.1 ... - Compact GL383568.1 ... - Compact GL383569.1 ... - Compact GL383570.1 ... - Compact GL383571.1 ... - Compact GL383572.1 ... - Compact GL383573.1 ... - Compact GL383574.1 ... - Compact GL383575.2 ... - Compact GL383576.1 ... - Compact GL383577.2 ... - Compact GL383578.2 ... - Compact GL383579.2 ... - Compact GL383580.2 ... - Compact GL383581.2 ... - Compact GL383582.2 ... - Compact GL383583.2 ... - Compact GL582966.2 ... - Compact GL877875.1 ... - Compact GL877876.1 ... - Compact GL949742.1 ... - Compact GL949746.1 ... - Compact GL949747.2 ... - Compact GL949748.2 ... - Compact GL949749.2 ... - Compact GL949750.2 ... - Compact GL949751.2 ... - Compact GL949752.1 ... - Compact GL949753.2 ... - Compact JH159136.1 ... - Compact JH159137.1 ... - Compact JH159146.1 ... - Compact JH159147.1 ... - Compact JH159148.1 ... - Compact JH636055.2 ... - Compact KB021644.2 ... - Compact KB663609.1 ... - Compact KI270302.1 ... - Compact KI270303.1 ... - Compact KI270304.1 ... - Compact KI270305.1 ... - Compact KI270310.1 ... - Compact KI270311.1 ... - Compact KI270312.1 ... - Compact KI270315.1 ... - Compact KI270316.1 ... - Compact KI270317.1 ... - Compact KI270320.1 ... - Compact KI270322.1 ... - Compact KI270329.1 ... - Compact KI270330.1 ... - Compact KI270333.1 ... - Compact KI270334.1 ... - Compact KI270335.1 ... - Compact KI270336.1 ... - Compact KI270337.1 ... - Compact KI270338.1 ... - Compact KI270340.1 ... - Compact KI270362.1 ... - Compact KI270363.1 ... - Compact KI270364.1 ... - Compact KI270366.1 ... - Compact KI270371.1 ... - Compact KI270372.1 ... - Compact KI270373.1 ... - Compact KI270374.1 ... - Compact KI270375.1 ... - Compact KI270376.1 ... - Compact KI270378.1 ... - Compact KI270379.1 ... - Compact KI270381.1 ... - Compact KI270382.1 ... - Compact KI270383.1 ... - Compact KI270384.1 ... - Compact KI270385.1 ... - Compact KI270386.1 ... - Compact KI270387.1 ... - Compact KI270388.1 ... - Compact KI270389.1 ... - Compact KI270390.1 ... - Compact KI270391.1 ... - Compact KI270392.1 ... - Compact KI270393.1 ... - Compact KI270394.1 ... - Compact KI270395.1 ... - Compact KI270396.1 ... - Compact KI270411.1 ... - Compact KI270412.1 ... - Compact KI270414.1 ... - Compact KI270417.1 ... - Compact KI270418.1 ... - Compact KI270419.1 ... - Compact KI270420.1 ... - Compact KI270422.1 ... - Compact KI270423.1 ... - Compact KI270424.1 ... - Compact KI270425.1 ... - Compact KI270429.1 ... - Compact KI270435.1 ... - Compact KI270438.1 ... - Compact KI270442.1 ... - Compact KI270448.1 ... - Compact KI270465.1 ... - Compact KI270466.1 ... - Compact KI270467.1 ... - Compact KI270468.1 ... - Compact KI270507.1 ... - Compact KI270508.1 ... - Compact KI270509.1 ... - Compact KI270510.1 ... - Compact KI270511.1 ... - Compact KI270512.1 ... - Compact KI270515.1 ... - Compact KI270516.1 ... - Compact KI270517.1 ... - Compact KI270518.1 ... - Compact KI270519.1 ... - Compact KI270521.1 ... - Compact KI270522.1 ... - Compact KI270528.1 ... - Compact KI270529.1 ... - Compact KI270530.1 ... - Compact KI270538.1 ... - Compact KI270539.1 ... - Compact KI270544.1 ... - Compact KI270548.1 ... - Compact KI270579.1 ... - Compact KI270580.1 ... - Compact KI270581.1 ... - Compact KI270582.1 ... - Compact KI270583.1 ... - Compact KI270584.1 ... - Compact KI270587.1 ... - Compact KI270588.1 ... - Compact KI270589.1 ... - Compact KI270590.1 ... - Compact KI270591.1 ... - Compact KI270593.1 ... - Compact KI270706.1 ... - Compact KI270707.1 ... - Compact KI270708.1 ... - Compact KI270709.1 ... - Compact KI270710.1 ... - Compact KI270711.1 ... - Write KI270711.1 to bigWig ... - Compact KI270712.1 ... - Compact KI270713.1 ... - Write KI270713.1 to bigWig ... - Compact KI270714.1 ... - Compact KI270715.1 ... - Compact KI270716.1 ... - Compact KI270717.1 ... - Compact KI270718.1 ... - Compact KI270719.1 ... - Compact KI270720.1 ... - Compact KI270721.1 ... - Write KI270721.1 to bigWig ... - Compact KI270722.1 ... - Compact KI270723.1 ... - Compact KI270724.1 ... - Compact KI270725.1 ... - Compact KI270726.1 ... - Write KI270726.1 to bigWig ... - Compact KI270727.1 ... - Compact KI270728.1 ... - Compact KI270729.1 ... - Compact KI270730.1 ... - Compact KI270731.1 ... - Compact KI270732.1 ... - Compact KI270733.1 ... - Compact KI270734.1 ... - Write KI270734.1 to bigWig ... - Compact KI270735.1 ... - Compact KI270736.1 ... - Compact KI270737.1 ... - Compact KI270738.1 ... - Compact KI270739.1 ... - Compact KI270740.1 ... - Compact KI270741.1 ... - Compact KI270742.1 ... - Compact KI270743.1 ... - Compact KI270744.1 ... - Compact KI270745.1 ... - Compact KI270746.1 ... - Compact KI270747.1 ... - Compact KI270748.1 ... - Compact KI270749.1 ... - Compact KI270750.1 ... - Compact KI270751.1 ... - Compact KI270752.1 ... - Compact KI270753.1 ... - Compact KI270754.1 ... - Compact KI270755.1 ... - Compact KI270756.1 ... - Compact KI270757.1 ... - Compact KI270758.1 ... - Compact KI270759.1 ... - Compact KI270760.1 ... - Compact KI270761.1 ... - Compact KI270762.1 ... - Compact KI270763.1 ... - Compact KI270764.1 ... - Compact KI270765.1 ... - Compact KI270766.1 ... - Compact KI270767.1 ... - Compact KI270768.1 ... - Compact KI270769.1 ... - Compact KI270770.1 ... - Compact KI270771.1 ... Add chromosome sizes to bigWig header Number of fragments: 1035898 Split fragments df by chromosome Calculate depth per chromosome: - GL000194.1 ... - GL000195.1 ... - GL000205.2 ... - GL000218.1 ... - GL000219.1 ... - KI270713.1 ... - KI270721.1 ... - KI270726.1 ... - chr1 ... - chr10 ... - chr11 ... - chr12 ... - chr13 ... - chr14 ... - chr15 ... - chr16 ... - chr17 ... - chr18 ... - chr19 ... - chr2 ... - chr20 ... - chr21 ... - chr22 ... - chr3 ... - chr4 ... - chr5 ... - chr6 ... - chr7 ... - chr8 ... - chr9 ... - chrX ... - chrY ... Compact depth array per chromosome (make ranges for consecutive the same values and remove zeros): - Compact GL000008.2 ... - Compact GL000009.2 ... - Compact GL000194.1 ... - Write GL000194.1 to bigWig ... - Compact GL000195.1 ... - Write GL000195.1 to bigWig ... - Compact GL000205.2 ... - Write GL000205.2 to bigWig ... - Compact GL000208.1 ... - Compact GL000209.2 ... - Compact GL000213.1 ... - Compact GL000214.1 ... - Compact GL000216.2 ... - Compact GL000218.1 ... - Write GL000218.1 to bigWig ... - Compact GL000219.1 ... - Write GL000219.1 to bigWig ... - Compact GL000220.1 ... - Compact GL000221.1 ... - Compact GL000224.1 ... - Compact GL000225.1 ... - Compact GL000226.1 ... - Compact GL000250.2 ... - Compact GL000251.2 ... - Compact GL000252.2 ... - Compact GL000253.2 ... - Compact GL000254.2 ... - Compact GL000255.2 ... - Compact GL000256.2 ... - Compact GL000257.2 ... - Compact GL000258.2 ... - Compact GL339449.2 ... - Compact GL383518.1 ... - Compact GL383519.1 ... - Compact GL383520.2 ... - Compact GL383521.1 ... - Compact GL383522.1 ... - Compact GL383526.1 ... - Compact GL383527.1 ... - Compact GL383528.1 ... - Compact GL383530.1 ... - Compact GL383531.1 ... - Compact GL383532.1 ... - Compact GL383533.1 ... - Compact GL383534.2 ... - Compact GL383539.1 ... - Compact GL383540.1 ... - Compact GL383541.1 ... - Compact GL383542.1 ... - Compact GL383545.1 ... - Compact GL383546.1 ... - Compact GL383547.1 ... - Compact GL383549.1 ... - Compact GL383550.2 ... - Compact GL383551.1 ... - Compact GL383552.1 ... - Compact GL383553.2 ... - Compact GL383554.1 ... - Compact GL383555.2 ... - Compact GL383556.1 ... - Compact GL383557.1 ... - Compact GL383563.3 ... - Compact GL383564.2 ... - Compact GL383565.1 ... - Compact GL383566.1 ... - Compact GL383567.1 ... - Compact GL383568.1 ... - Compact GL383569.1 ... - Compact GL383570.1 ... - Compact GL383571.1 ... - Compact GL383572.1 ... - Compact GL383573.1 ... - Compact GL383574.1 ... - Compact GL383575.2 ... - Compact GL383576.1 ... - Compact GL383577.2 ... - Compact GL383578.2 ... - Compact GL383579.2 ... - Compact GL383580.2 ... - Compact GL383581.2 ... - Compact GL383582.2 ... - Compact GL383583.2 ... - Compact GL582966.2 ... - Compact GL877875.1 ... - Compact GL877876.1 ... - Compact GL949742.1 ... - Compact GL949746.1 ... - Compact GL949747.2 ... - Compact GL949748.2 ... - Compact GL949749.2 ... - Compact GL949750.2 ... - Compact GL949751.2 ... - Compact GL949752.1 ... - Compact GL949753.2 ... - Compact JH159136.1 ... - Compact JH159137.1 ... - Compact JH159146.1 ... - Compact JH159147.1 ... - Compact JH159148.1 ... - Compact JH636055.2 ... - Compact KB021644.2 ... - Compact KB663609.1 ... - Compact KI270302.1 ... - Compact KI270303.1 ... - Compact KI270304.1 ... - Compact KI270305.1 ... - Compact KI270310.1 ... - Compact KI270311.1 ... - Compact KI270312.1 ... - Compact KI270315.1 ... - Compact KI270316.1 ... - Compact KI270317.1 ... - Compact KI270320.1 ... - Compact KI270322.1 ... - Compact KI270329.1 ... - Compact KI270330.1 ... - Compact KI270333.1 ... - Compact KI270334.1 ... - Compact KI270335.1 ... - Compact KI270336.1 ... - Compact KI270337.1 ... - Compact KI270338.1 ... - Compact KI270340.1 ... - Compact KI270362.1 ... - Compact KI270363.1 ... - Compact KI270364.1 ... - Compact KI270366.1 ... - Compact KI270371.1 ... - Compact KI270372.1 ... - Compact KI270373.1 ... - Compact KI270374.1 ... - Compact KI270375.1 ... - Compact KI270376.1 ... - Compact KI270378.1 ... - Compact KI270379.1 ... - Compact KI270381.1 ... - Compact KI270382.1 ... - Compact KI270383.1 ... - Compact KI270384.1 ... - Compact KI270385.1 ... - Compact KI270386.1 ... - Compact KI270387.1 ... - Compact KI270388.1 ... - Compact KI270389.1 ... - Compact KI270390.1 ... - Compact KI270391.1 ... - Compact KI270392.1 ... - Compact KI270393.1 ... - Compact KI270394.1 ... - Compact KI270395.1 ... - Compact KI270396.1 ... - Compact KI270411.1 ... - Compact KI270412.1 ... - Compact KI270414.1 ... - Compact KI270417.1 ... - Compact KI270418.1 ... - Compact KI270419.1 ... - Compact KI270420.1 ... - Compact KI270422.1 ... - Compact KI270423.1 ... - Compact KI270424.1 ... - Compact KI270425.1 ... - Compact KI270429.1 ... - Compact KI270435.1 ... - Compact KI270438.1 ... - Compact KI270442.1 ... - Compact KI270448.1 ... - Compact KI270465.1 ... - Compact KI270466.1 ... - Compact KI270467.1 ... - Compact KI270468.1 ... - Compact KI270507.1 ... - Compact KI270508.1 ... - Compact KI270509.1 ... - Compact KI270510.1 ... - Compact KI270511.1 ... - Compact KI270512.1 ... - Compact KI270515.1 ... - Compact KI270516.1 ... - Compact KI270517.1 ... - Compact KI270518.1 ... - Compact KI270519.1 ... - Compact KI270521.1 ... - Compact KI270522.1 ... - Compact KI270528.1 ... - Compact KI270529.1 ... - Compact KI270530.1 ... - Compact KI270538.1 ... - Compact KI270539.1 ... - Compact KI270544.1 ... - Compact KI270548.1 ... - Compact KI270579.1 ... - Compact KI270580.1 ... - Compact KI270581.1 ... - Compact KI270582.1 ... - Compact KI270583.1 ... - Compact KI270584.1 ... - Compact KI270587.1 ... - Compact KI270588.1 ... - Compact KI270589.1 ... - Compact KI270590.1 ... - Compact KI270591.1 ... - Compact KI270593.1 ... - Compact KI270706.1 ... - Compact KI270707.1 ... - Compact KI270708.1 ... - Compact KI270709.1 ... - Compact KI270710.1 ... - Compact KI270711.1 ... - Compact KI270712.1 ... - Compact KI270713.1 ... - Write KI270713.1 to bigWig ... - Compact KI270714.1 ... - Compact KI270715.1 ... - Compact KI270716.1 ... - Compact KI270717.1 ... - Compact KI270718.1 ... - Compact KI270719.1 ... - Compact KI270720.1 ... - Compact KI270721.1 ... - Write KI270721.1 to bigWig ... - Compact KI270722.1 ... - Compact KI270723.1 ... - Compact KI270724.1 ... - Compact KI270725.1 ... - Compact KI270726.1 ... - Write KI270726.1 to bigWig ... - Compact KI270727.1 ... - Compact KI270728.1 ... - Compact KI270729.1 ... - Compact KI270730.1 ... - Compact KI270731.1 ... - Compact KI270732.1 ... - Compact KI270733.1 ... - Compact KI270734.1 ... - Compact KI270735.1 ... - Compact KI270736.1 ... - Compact KI270737.1 ... - Compact KI270738.1 ... - Compact KI270739.1 ... - Compact KI270740.1 ... - Compact KI270741.1 ... - Compact KI270742.1 ... - Compact KI270743.1 ... - Compact KI270744.1 ... - Compact KI270745.1 ... - Compact KI270746.1 ... - Compact KI270747.1 ... - Compact KI270748.1 ... - Compact KI270749.1 ... - Compact KI270750.1 ... - Compact KI270751.1 ... - Compact KI270752.1 ... - Compact KI270753.1 ... - Compact KI270754.1 ... - Compact KI270755.1 ... - Compact KI270756.1 ... - Compact KI270757.1 ... - Compact KI270758.1 ... - Compact KI270759.1 ... - Compact KI270760.1 ... - Compact KI270761.1 ... - Compact KI270762.1 ... - Compact KI270763.1 ... - Compact KI270764.1 ... - Compact KI270765.1 ... - Compact KI270766.1 ... - Compact KI270767.1 ... - Compact KI270768.1 ... - Compact KI270769.1 ... - Compact KI270770.1 ... - Compact KI270771.1 ... - Compact KI270772.1 ... - Compact KI270773.1 ... Add chromosome sizes to bigWig header Number of fragments: 3409287 Split fragments df by chromosome Calculate depth per chromosome: - GL000009.2 ... - GL000194.1 ... - GL000195.1 ... - GL000205.2 ... - GL000213.1 ... - GL000218.1 ... - GL000219.1 ... - KI270713.1 ... - KI270721.1 ... - KI270726.1 ... - KI270734.1 ... - chr1 ... - chr10 ... - chr11 ... - chr12 ... - chr13 ... - chr14 ... - chr15 ... - chr16 ... - chr17 ... - chr18 ... - chr19 ... - chr2 ... - chr20 ... - chr21 ... - chr22 ... - chr3 ... - chr4 ... - chr5 ... - chr6 ... - chr7 ... - chr8 ... - chr9 ... - chrX ... - chrY ... Compact depth array per chromosome (make ranges for consecutive the same values and remove zeros): - Compact GL000008.2 ... - Compact GL000009.2 ... - Write GL000009.2 to bigWig ... - Compact GL000194.1 ... - Write GL000194.1 to bigWig ... - Compact GL000195.1 ... - Write GL000195.1 to bigWig ... - Compact GL000205.2 ... - Write GL000205.2 to bigWig ... - Compact GL000208.1 ... - Compact GL000209.2 ... - Compact GL000213.1 ... - Write GL000213.1 to bigWig ... - Compact GL000214.1 ... - Compact GL000216.2 ... - Compact GL000218.1 ... - Write GL000218.1 to bigWig ... - Compact GL000219.1 ... - Write GL000219.1 to bigWig ... - Compact GL000220.1 ... - Compact GL000221.1 ... - Compact GL000224.1 ... - Compact GL000225.1 ... - Compact GL000226.1 ... - Compact GL000250.2 ... - Compact GL000251.2 ... - Compact GL000252.2 ... - Compact GL000253.2 ... - Compact GL000254.2 ... - Compact GL000255.2 ... - Compact GL000256.2 ... - Compact GL000257.2 ... - Compact GL000258.2 ... - Compact GL339449.2 ... - Compact GL383518.1 ... - Compact GL383519.1 ... - Compact GL383520.2 ... - Compact GL383521.1 ... - Compact GL383522.1 ... - Compact GL383526.1 ... - Compact GL383527.1 ... - Compact GL383528.1 ... - Compact GL383530.1 ... - Compact GL383531.1 ... - Compact GL383532.1 ... - Compact GL383533.1 ... - Compact GL383534.2 ... - Compact GL383539.1 ... - Compact GL383540.1 ... - Compact GL383541.1 ... - Compact GL383542.1 ... - Compact GL383545.1 ... - Compact GL383546.1 ... - Compact GL383547.1 ... - Compact GL383549.1 ... - Compact GL383550.2 ... - Compact GL383551.1 ... - Compact GL383552.1 ... - Compact GL383553.2 ... - Compact GL383554.1 ... - Compact GL383555.2 ... - Compact GL383556.1 ... - Compact GL383557.1 ... - Compact GL383563.3 ... - Compact GL383564.2 ... - Compact GL383565.1 ... - Compact GL383566.1 ... - Compact GL383567.1 ... - Compact GL383568.1 ... - Compact GL383569.1 ... - Compact GL383570.1 ... - Compact GL383571.1 ... - Compact GL383572.1 ... - Compact GL383573.1 ... - Compact GL383574.1 ... - Compact GL383575.2 ... - Compact GL383576.1 ... - Compact GL383577.2 ... - Compact GL383578.2 ... - Compact GL383579.2 ... - Compact GL383580.2 ... - Compact GL383581.2 ... - Compact GL383582.2 ... - Compact GL383583.2 ... - Compact GL582966.2 ... - Compact GL877875.1 ... - Compact GL877876.1 ... - Compact GL949742.1 ... - Compact GL949746.1 ... - Compact GL949747.2 ... - Compact GL949748.2 ... - Compact GL949749.2 ... - Compact GL949750.2 ... - Compact GL949751.2 ... - Compact GL949752.1 ... - Compact GL949753.2 ... - Compact JH159136.1 ... - Compact JH159137.1 ... - Compact JH159146.1 ... - Compact JH159147.1 ... - Compact JH159148.1 ... - Compact JH636055.2 ... - Compact KB021644.2 ... - Compact KB663609.1 ... - Compact KI270302.1 ... - Compact KI270303.1 ... - Compact KI270304.1 ... - Compact KI270305.1 ... - Compact KI270310.1 ... - Compact KI270311.1 ... - Compact KI270312.1 ... - Compact KI270315.1 ... - Compact KI270316.1 ... - Compact KI270317.1 ... - Compact KI270320.1 ... - Compact KI270322.1 ... - Compact KI270329.1 ... - Compact KI270330.1 ... - Compact KI270333.1 ... - Compact KI270334.1 ... - Compact KI270335.1 ... - Compact KI270336.1 ... - Compact KI270337.1 ... - Compact KI270338.1 ... - Compact KI270340.1 ... - Compact KI270362.1 ... - Compact KI270363.1 ... - Compact KI270364.1 ... - Compact KI270366.1 ... - Compact KI270371.1 ... - Compact KI270372.1 ... - Compact KI270373.1 ... - Compact KI270374.1 ... - Compact KI270375.1 ... - Compact KI270376.1 ... - Compact KI270378.1 ... - Compact KI270379.1 ... - Compact KI270381.1 ... - Compact KI270382.1 ... - Compact KI270383.1 ... - Compact KI270384.1 ... - Compact KI270385.1 ... - Compact KI270386.1 ... - Compact KI270387.1 ... - Compact KI270388.1 ... - Compact KI270389.1 ... - Compact KI270390.1 ... - Compact KI270391.1 ... - Compact KI270392.1 ... - Compact KI270393.1 ... - Compact KI270394.1 ... - Compact KI270395.1 ... - Compact KI270396.1 ... - Compact KI270411.1 ... - Compact KI270412.1 ... - Compact KI270414.1 ... - Compact KI270417.1 ... - Compact KI270418.1 ... - Compact KI270419.1 ... - Compact KI270420.1 ... - Compact KI270422.1 ... - Compact KI270423.1 ... - Compact KI270424.1 ... - Compact KI270425.1 ... - Compact KI270429.1 ... - Compact KI270435.1 ... - Compact KI270438.1 ... - Compact KI270442.1 ... - Compact KI270448.1 ... - Compact KI270465.1 ... - Compact KI270466.1 ... - Compact KI270467.1 ... - Compact KI270468.1 ... - Compact KI270507.1 ... - Compact KI270508.1 ... - Compact KI270509.1 ... - Compact KI270510.1 ... - Compact KI270511.1 ... - Compact KI270512.1 ... - Compact KI270515.1 ... - Compact KI270516.1 ... - Compact KI270517.1 ... - Compact KI270518.1 ... - Compact KI270519.1 ... - Compact KI270521.1 ... - Compact KI270522.1 ... - Compact KI270528.1 ... - Compact KI270529.1 ... - Compact KI270530.1 ... - Compact KI270538.1 ... - Compact KI270539.1 ... - Compact KI270544.1 ... - Compact KI270548.1 ... - Compact KI270579.1 ... - Compact KI270580.1 ... - Compact KI270581.1 ... - Compact KI270582.1 ... - Compact KI270583.1 ... - Compact KI270584.1 ... - Compact KI270587.1 ... - Compact KI270588.1 ... - Compact KI270589.1 ... - Compact KI270590.1 ... - Compact KI270591.1 ... - Compact KI270593.1 ... - Compact KI270706.1 ... - Compact KI270707.1 ... - Compact KI270708.1 ... - Compact KI270709.1 ... - Compact KI270710.1 ... - Compact KI270711.1 ... - Compact KI270712.1 ... - Compact KI270713.1 ... - Write KI270713.1 to bigWig ... - Compact KI270714.1 ... - Compact KI270715.1 ... - Compact KI270716.1 ... - Compact KI270717.1 ... - Compact KI270718.1 ... - Compact KI270719.1 ... - Compact KI270720.1 ... - Compact KI270721.1 ... - Write KI270721.1 to bigWig ... - Compact KI270722.1 ... - Compact KI270723.1 ... - Compact KI270724.1 ... - Compact KI270725.1 ... - Compact KI270726.1 ... - Write KI270726.1 to bigWig ... - Compact KI270727.1 ... - Compact KI270728.1 ... - Compact KI270729.1 ... - Compact KI270730.1 ... - Compact KI270731.1 ... - Compact KI270732.1 ... - Compact KI270733.1 ... - Compact KI270734.1 ... - Write KI270734.1 to bigWig ... - Compact KI270735.1 ... - Compact KI270736.1 ... - Compact KI270737.1 ... - Compact KI270738.1 ... - Compact KI270739.1 ... - Compact KI270740.1 ... - Compact KI270741.1 ... - Compact KI270742.1 ... - Compact KI270743.1 ... - Compact KI270744.1 ... - Compact KI270745.1 ... - Compact KI270746.1 ... - Compact KI270747.1 ... - Compact KI270748.1 ... - Compact KI270749.1 ... - Compact KI270750.1 ... - Compact KI270751.1 ... - Compact KI270752.1 ... - Compact KI270753.1 ... - Compact KI270754.1 ... - Compact KI270755.1 ... - Compact KI270756.1 ... - Compact KI270757.1 ... - Compact KI270758.1 ... - Compact KI270759.1 ... - Compact KI270760.1 ... - Compact KI270761.1 ... - Compact KI270762.1 ... - Compact KI270763.1 ... - Compact KI270764.1 ... - Compact KI270765.1 ... - Compact KI270766.1 ... - Compact KI270767.1 ... Add chromosome sizes to bigWig header Number of fragments: 11810322 Split fragments df by chromosome Calculate depth per chromosome: - GL000009.2 ... - GL000194.1 ... - GL000195.1 ... - GL000205.2 ... - GL000213.1 ... - GL000218.1 ... - GL000219.1 ... - KI270711.1 ... - KI270713.1 ... - KI270721.1 ... - KI270726.1 ... - KI270727.1 ... - KI270728.1 ... - KI270731.1 ... - KI270734.1 ... - chr1 ... - chr10 ... - chr11 ... - chr12 ... - chr13 ... - chr14 ... - chr15 ... - chr16 ... - chr17 ... - chr18 ... - chr19 ... - chr2 ... - chr20 ... - chr21 ... - chr22 ... - chr3 ... - chr4 ... - chr5 ... - chr6 ... - chr7 ... - chr8 ... - chr9 ... - chrX ... - chrY ... Compact depth array per chromosome (make ranges for consecutive the same values and remove zeros): - Compact GL000008.2 ... - Compact GL000009.2 ... - Write GL000009.2 to bigWig ... - Compact GL000194.1 ... - Write GL000194.1 to bigWig ... - Compact GL000195.1 ... - Write GL000195.1 to bigWig ... - Compact GL000205.2 ... - Write GL000205.2 to bigWig ... - Compact GL000208.1 ... - Compact GL000209.2 ... - Compact GL000213.1 ... - Write GL000213.1 to bigWig ... - Compact GL000214.1 ... - Compact GL000216.2 ... - Compact GL000218.1 ... - Write GL000218.1 to bigWig ... - Compact GL000219.1 ... - Write GL000219.1 to bigWig ... - Compact GL000220.1 ... - Compact GL000221.1 ... - Compact GL000224.1 ... - Compact GL000225.1 ... - Compact GL000226.1 ... - Compact GL000250.2 ... - Compact GL000251.2 ... - Compact GL000252.2 ... - Compact GL000253.2 ... - Compact GL000254.2 ... - Compact GL000255.2 ... - Compact GL000256.2 ... - Compact GL000257.2 ... - Compact GL000258.2 ... - Compact GL339449.2 ... - Compact GL383518.1 ... - Compact GL383519.1 ... - Compact GL383520.2 ... - Compact GL383521.1 ... - Compact GL383522.1 ... - Compact GL383526.1 ... - Compact GL383527.1 ... - Compact GL383528.1 ... - Compact GL383530.1 ... - Compact GL383531.1 ... - Compact GL383532.1 ... - Compact GL383533.1 ... - Compact GL383534.2 ... - Compact GL383539.1 ... - Compact GL383540.1 ... - Compact GL383541.1 ... - Compact GL383542.1 ... - Compact GL383545.1 ... - Compact GL383546.1 ... - Compact GL383547.1 ... - Compact GL383549.1 ... - Compact GL383550.2 ... - Compact GL383551.1 ... - Compact GL383552.1 ... - Compact GL383553.2 ... - Compact GL383554.1 ... - Compact GL383555.2 ... - Compact GL383556.1 ... - Compact GL383557.1 ... - Compact GL383563.3 ... - Compact GL383564.2 ... - Compact GL383565.1 ... - Compact GL383566.1 ... - Compact GL383567.1 ... - Compact GL383568.1 ... - Compact GL383569.1 ... - Compact GL383570.1 ... - Compact GL383571.1 ... - Compact GL383572.1 ... - Compact GL383573.1 ... - Compact GL383574.1 ... - Compact GL383575.2 ... - Compact GL383576.1 ... - Compact GL383577.2 ... - Compact GL383578.2 ... - Compact GL383579.2 ... - Compact GL383580.2 ... - Compact GL383581.2 ... - Compact GL383582.2 ... - Compact GL383583.2 ... - Compact GL582966.2 ... - Compact GL877875.1 ... - Compact GL877876.1 ... - Compact GL949742.1 ... - Compact GL949746.1 ... - Compact GL949747.2 ... - Compact GL949748.2 ... - Compact GL949749.2 ... - Compact GL949750.2 ... - Compact GL949751.2 ... - Compact GL949752.1 ... - Compact GL949753.2 ... - Compact JH159136.1 ... - Compact JH159137.1 ... - Compact JH159146.1 ... - Compact JH159147.1 ... - Compact JH159148.1 ... - Compact JH636055.2 ... - Compact KB021644.2 ... - Compact KB663609.1 ... - Compact KI270302.1 ... - Compact KI270303.1 ... - Compact KI270304.1 ... - Compact KI270305.1 ... - Compact KI270310.1 ... - Compact KI270311.1 ... - Compact KI270312.1 ... - Compact KI270315.1 ... - Compact KI270316.1 ... - Compact KI270317.1 ... - Compact KI270320.1 ... - Compact KI270322.1 ... - Compact KI270329.1 ... - Compact KI270330.1 ... - Compact KI270333.1 ... - Compact KI270334.1 ... - Compact KI270335.1 ... - Compact KI270336.1 ... - Compact KI270337.1 ... - Compact KI270338.1 ... - Compact KI270340.1 ... - Compact KI270362.1 ... - Compact KI270363.1 ... - Compact KI270364.1 ... - Compact KI270366.1 ... - Compact KI270371.1 ... - Compact KI270372.1 ... - Compact KI270373.1 ... - Compact KI270374.1 ... - Compact KI270375.1 ... - Compact KI270376.1 ... - Compact KI270378.1 ... - Compact KI270379.1 ... - Compact KI270381.1 ... - Compact KI270382.1 ... - Compact KI270383.1 ... - Compact KI270384.1 ... - Compact KI270385.1 ... - Compact KI270386.1 ... - Compact KI270387.1 ... - Compact KI270388.1 ... - Compact KI270389.1 ... - Compact KI270390.1 ... - Compact KI270391.1 ... - Compact KI270392.1 ... - Compact KI270393.1 ... - Compact KI270394.1 ... - Compact KI270395.1 ... - Compact KI270396.1 ... - Compact KI270411.1 ... - Compact KI270412.1 ... - Compact KI270414.1 ... - Compact KI270417.1 ... - Compact KI270418.1 ... - Compact KI270419.1 ... - Compact KI270420.1 ... - Compact KI270422.1 ... - Compact KI270423.1 ... - Compact KI270424.1 ... - Compact KI270425.1 ... - Compact KI270429.1 ... - Compact KI270435.1 ... - Compact KI270438.1 ... - Compact KI270442.1 ... - Compact KI270448.1 ... - Compact KI270465.1 ... - Compact KI270466.1 ... - Compact KI270467.1 ... - Compact KI270468.1 ... - Compact KI270507.1 ... - Compact KI270508.1 ... - Compact KI270509.1 ... - Compact KI270510.1 ... - Compact KI270511.1 ... - Compact KI270512.1 ... - Compact KI270515.1 ... - Compact KI270516.1 ... - Compact KI270517.1 ... - Compact KI270518.1 ... - Compact KI270519.1 ... - Compact KI270521.1 ... - Compact KI270522.1 ... - Compact KI270528.1 ... - Compact KI270529.1 ... - Compact KI270530.1 ... - Compact KI270538.1 ... - Compact KI270539.1 ... - Compact KI270544.1 ... - Compact KI270548.1 ... - Compact KI270579.1 ... - Compact KI270580.1 ... - Compact KI270581.1 ... - Compact KI270582.1 ... - Compact KI270583.1 ... - Compact KI270584.1 ... - Compact KI270587.1 ... - Compact KI270588.1 ... - Compact KI270589.1 ... - Compact KI270590.1 ... - Compact KI270591.1 ... - Compact KI270593.1 ... - Compact KI270706.1 ... - Compact KI270707.1 ... - Compact KI270708.1 ... - Compact KI270709.1 ... - Compact KI270710.1 ... - Compact KI270711.1 ... - Write KI270711.1 to bigWig ... - Compact KI270712.1 ... - Compact KI270713.1 ... - Write KI270713.1 to bigWig ... - Compact KI270714.1 ... - Compact KI270715.1 ... - Compact KI270716.1 ... - Compact KI270717.1 ... - Compact KI270718.1 ... - Compact KI270719.1 ... - Compact KI270720.1 ... - Compact KI270721.1 ... - Write KI270721.1 to bigWig ... - Compact KI270722.1 ... - Compact KI270723.1 ... - Compact KI270724.1 ... - Compact KI270725.1 ... - Compact KI270726.1 ... - Write KI270726.1 to bigWig ... - Compact KI270727.1 ... - Write KI270727.1 to bigWig ... - Compact KI270728.1 ... - Write KI270728.1 to bigWig ... - Compact KI270729.1 ... - Compact KI270730.1 ... - Compact KI270731.1 ... - Write KI270731.1 to bigWig ... - Compact KI270732.1 ... - Compact KI270733.1 ... - Compact KI270734.1 ... - Write KI270734.1 to bigWig ... - Compact KI270735.1 ... - Compact KI270736.1 ... - Compact KI270737.1 ... - Compact KI270738.1 ... - Compact KI270739.1 ... - Compact KI270740.1 ... - Compact KI270741.1 ... - Compact KI270742.1 ... - Compact KI270743.1 ... - Compact KI270744.1 ... - Compact KI270745.1 ... - Compact KI270746.1 ... - Compact KI270747.1 ... - Compact KI270748.1 ... - Compact KI270749.1 ... - Compact KI270750.1 ... - Compact KI270751.1 ... - Compact KI270752.1 ... - Compact KI270753.1 ... - Compact KI270754.1 ... - Compact KI270755.1 ... - Compact KI270756.1 ... - Compact KI270757.1 ... - Compact KI270758.1 ... - Compact KI270759.1 ... Add chromosome sizes to bigWig header Number of fragments: 3052649 Split fragments df by chromosome Calculate depth per chromosome: - GL000009.2 ... - GL000194.1 ... - GL000195.1 ... - GL000205.2 ... - GL000218.1 ... - GL000219.1 ... - KI270711.1 ... - KI270713.1 ... - KI270721.1 ... - KI270726.1 ... - KI270727.1 ... - KI270728.1 ... - KI270731.1 ... - KI270734.1 ... - chr1 ... - chr10 ... - chr11 ... - chr12 ... - chr13 ... - chr14 ... - chr15 ... - chr16 ... - chr17 ... - chr18 ... - chr19 ... - chr2 ... - chr20 ... - chr21 ... - chr22 ... - chr3 ... - chr4 ... - chr5 ... - chr6 ... - chr7 ... - chr8 ... - chr9 ... - chrX ... - chrY ... Compact depth array per chromosome (make ranges for consecutive the same values and remove zeros): - Compact GL000008.2 ... - Compact GL000009.2 ... - Write GL000009.2 to bigWig ... - Compact GL000194.1 ... - Write GL000194.1 to bigWig ... - Compact GL000195.1 ... - Write GL000195.1 to bigWig ... - Compact GL000205.2 ... - Write GL000205.2 to bigWig ... - Compact GL000208.1 ... - Compact GL000209.2 ... - Compact GL000213.1 ... - Compact GL000214.1 ... - Compact GL000216.2 ... - Compact GL000218.1 ... - Write GL000218.1 to bigWig ... - Compact GL000219.1 ... - Write GL000219.1 to bigWig ... - Compact GL000220.1 ... - Compact GL000221.1 ... - Compact GL000224.1 ... - Compact GL000225.1 ... - Compact GL000226.1 ... - Compact GL000250.2 ... - Compact GL000251.2 ... - Compact GL000252.2 ... - Compact GL000253.2 ... - Compact GL000254.2 ... - Compact GL000255.2 ... - Compact GL000256.2 ... - Compact GL000257.2 ... - Compact GL000258.2 ... - Compact GL339449.2 ... - Compact GL383518.1 ... - Compact GL383519.1 ... - Compact GL383520.2 ... - Compact GL383521.1 ... - Compact GL383522.1 ... - Compact GL383526.1 ... - Compact GL383527.1 ... - Compact GL383528.1 ... - Compact GL383530.1 ... - Compact GL383531.1 ... - Compact GL383532.1 ... - Compact GL383533.1 ... - Compact GL383534.2 ... - Compact GL383539.1 ... - Compact GL383540.1 ... - Compact GL383541.1 ... - Compact GL383542.1 ... - Compact GL383545.1 ... - Compact GL383546.1 ... - Compact GL383547.1 ... - Compact GL383549.1 ... - Compact GL383550.2 ... - Compact GL383551.1 ... - Compact GL383552.1 ... - Compact GL383553.2 ... - Compact GL383554.1 ... - Compact GL383555.2 ... - Compact GL383556.1 ... - Compact GL383557.1 ... - Compact GL383563.3 ... - Compact GL383564.2 ... - Compact GL383565.1 ... - Compact GL383566.1 ... - Compact GL383567.1 ... - Compact GL383568.1 ... - Compact GL383569.1 ... - Compact GL383570.1 ... - Compact GL383571.1 ... - Compact GL383572.1 ... - Compact GL383573.1 ... - Compact GL383574.1 ... - Compact GL383575.2 ... - Compact GL383576.1 ... - Compact GL383577.2 ... - Compact GL383578.2 ... - Compact GL383579.2 ... - Compact GL383580.2 ... - Compact GL383581.2 ... - Compact GL383582.2 ... - Compact GL383583.2 ... - Compact GL582966.2 ... - Compact GL877875.1 ... - Compact GL877876.1 ... - Compact GL949742.1 ... - Compact GL949746.1 ... - Compact GL949747.2 ... - Compact GL949748.2 ... - Compact GL949749.2 ... - Compact GL949750.2 ... - Compact GL949751.2 ... - Compact GL949752.1 ... - Compact GL949753.2 ... - Compact JH159136.1 ... - Compact JH159137.1 ... - Compact JH159146.1 ... - Compact JH159147.1 ... - Compact JH159148.1 ... - Compact JH636055.2 ... - Compact KB021644.2 ... - Compact KB663609.1 ... - Compact KI270302.1 ... - Compact KI270303.1 ... - Compact KI270304.1 ... - Compact KI270305.1 ... - Compact KI270310.1 ... - Compact KI270311.1 ... - Compact KI270312.1 ... - Compact KI270315.1 ... - Compact KI270316.1 ... - Compact KI270317.1 ... - Compact KI270320.1 ... - Compact KI270322.1 ... - Compact KI270329.1 ... - Compact KI270330.1 ... - Compact KI270333.1 ... - Compact KI270334.1 ... - Compact KI270335.1 ... - Compact KI270336.1 ... - Compact KI270337.1 ... - Compact KI270338.1 ... - Compact KI270340.1 ... - Compact KI270362.1 ... - Compact KI270363.1 ... - Compact KI270364.1 ... - Compact KI270366.1 ... - Compact KI270371.1 ... - Compact KI270372.1 ... - Compact KI270373.1 ... - Compact KI270374.1 ... - Compact KI270375.1 ... - Compact KI270376.1 ... - Compact KI270378.1 ... - Compact KI270379.1 ... - Compact KI270381.1 ... - Compact KI270382.1 ... - Compact KI270383.1 ... - Compact KI270384.1 ... - Compact KI270385.1 ... - Compact KI270386.1 ... - Compact KI270387.1 ... - Compact KI270388.1 ... - Compact KI270389.1 ... - Compact KI270390.1 ... - Compact KI270391.1 ... - Compact KI270392.1 ... - Compact KI270393.1 ... - Compact KI270394.1 ... - Compact KI270395.1 ... - Compact KI270396.1 ... - Compact KI270411.1 ... - Compact KI270412.1 ... - Compact KI270414.1 ... - Compact KI270417.1 ... - Compact KI270418.1 ... - Compact KI270419.1 ... - Compact KI270420.1 ... - Compact KI270422.1 ... - Compact KI270423.1 ... - Compact KI270424.1 ... - Compact KI270425.1 ... - Compact KI270429.1 ... - Compact KI270435.1 ... - Compact KI270438.1 ... - Compact KI270442.1 ... - Compact KI270448.1 ... - Compact KI270465.1 ... - Compact KI270466.1 ... - Compact KI270467.1 ... - Compact KI270468.1 ... - Compact KI270507.1 ... - Compact KI270508.1 ... - Compact KI270509.1 ... - Compact KI270510.1 ... - Compact KI270511.1 ... - Compact KI270512.1 ... - Compact KI270515.1 ... - Compact KI270516.1 ... - Compact KI270517.1 ... - Compact KI270518.1 ... - Compact KI270519.1 ... - Compact KI270521.1 ... - Compact KI270522.1 ... - Compact KI270528.1 ... - Compact KI270529.1 ... - Compact KI270530.1 ... - Compact KI270538.1 ... - Compact KI270539.1 ... - Compact KI270544.1 ... - Compact KI270548.1 ... - Compact KI270579.1 ... - Compact KI270580.1 ... - Compact KI270581.1 ... - Compact KI270582.1 ... - Compact KI270583.1 ... - Compact KI270584.1 ... - Compact KI270587.1 ... - Compact KI270588.1 ... - Compact KI270589.1 ... - Compact KI270590.1 ... - Compact KI270591.1 ... - Compact KI270593.1 ... - Compact KI270706.1 ... - Compact KI270707.1 ... - Compact KI270708.1 ... - Compact KI270709.1 ... - Compact KI270710.1 ... - Compact KI270711.1 ... - Write KI270711.1 to bigWig ... - Compact KI270712.1 ... - Compact KI270713.1 ... - Write KI270713.1 to bigWig ... - Compact KI270714.1 ... - Compact KI270715.1 ... - Compact KI270716.1 ... - Compact KI270717.1 ... - Compact KI270718.1 ... - Compact KI270719.1 ... - Compact KI270720.1 ... - Compact KI270721.1 ... - Write KI270721.1 to bigWig ... - Compact KI270722.1 ... - Compact KI270723.1 ... - Compact KI270724.1 ... - Compact KI270725.1 ... - Compact KI270726.1 ... - Write KI270726.1 to bigWig ... - Compact KI270727.1 ... - Write KI270727.1 to bigWig ... - Compact KI270728.1 ... - Write KI270728.1 to bigWig ... - Compact KI270729.1 ... - Compact KI270730.1 ... - Compact KI270731.1 ... - Write KI270731.1 to bigWig ... - Compact KI270732.1 ... - Compact KI270733.1 ... - Compact KI270734.1 ... - Write KI270734.1 to bigWig ... - Compact KI270735.1 ... - Compact KI270736.1 ... - Compact KI270737.1 ... - Compact KI270738.1 ... - Compact KI270739.1 ... - Compact KI270740.1 ... - Compact KI270741.1 ... - Compact KI270742.1 ... - Compact KI270743.1 ... - Compact KI270744.1 ... - Compact KI270745.1 ... - Compact KI270746.1 ... - Compact KI270747.1 ... - Compact KI270748.1 ... - Compact KI270749.1 ... - Compact KI270750.1 ... - Compact KI270751.1 ... - Compact KI270752.1 ... - Compact KI270753.1 ... - Compact KI270754.1 ... - Compact KI270755.1 ... - Compact KI270756.1 ... - Compact KI270757.1 ... - Compact KI270758.1 ... - Compact KI270759.1 ... - Compact KI270760.1 ... - Compact KI270761.1 ... Add chromosome sizes to bigWig header Number of fragments: 14387690 Split fragments df by chromosome Calculate depth per chromosome: - GL000009.2 ... - GL000194.1 ... - GL000195.1 ... - GL000205.2 ... - GL000218.1 ... - GL000219.1 ... - KI270711.1 ... - KI270713.1 ... - KI270721.1 ... - KI270726.1 ... - KI270727.1 ... - KI270728.1 ... - KI270731.1 ... - KI270734.1 ... - chr1 ... - chr10 ... - chr11 ... - chr12 ... - chr13 ... - chr14 ... - chr15 ... - chr16 ... - chr17 ... - chr18 ... - chr19 ... - chr2 ... - chr20 ... - chr21 ... - chr22 ... - chr3 ... - chr4 ... - chr5 ... - chr6 ... - chr7 ... - chr8 ... - chr9 ... - chrX ... - chrY ... Compact depth array per chromosome (make ranges for consecutive the same values and remove zeros): - Compact GL000008.2 ... - Compact GL000009.2 ... - Write GL000009.2 to bigWig ... - Compact GL000194.1 ... - Write GL000194.1 to bigWig ... - Compact GL000195.1 ... - Write GL000195.1 to bigWig ... - Compact GL000205.2 ... - Write GL000205.2 to bigWig ... - Compact GL000208.1 ... - Compact GL000209.2 ... - Compact GL000213.1 ... - Compact GL000214.1 ... - Compact GL000216.2 ... - Compact GL000218.1 ... - Write GL000218.1 to bigWig ... - Compact GL000219.1 ... - Write GL000219.1 to bigWig ... - Compact GL000220.1 ... - Compact GL000221.1 ... - Compact GL000224.1 ... - Compact GL000225.1 ... - Compact GL000226.1 ... - Compact GL000250.2 ... - Compact GL000251.2 ... - Compact GL000252.2 ... - Compact GL000253.2 ... - Compact GL000254.2 ... - Compact GL000255.2 ... - Compact GL000256.2 ... - Compact GL000257.2 ... - Compact GL000258.2 ... - Compact GL339449.2 ... - Compact GL383518.1 ... - Compact GL383519.1 ... - Compact GL383520.2 ... - Compact GL383521.1 ... - Compact GL383522.1 ... - Compact GL383526.1 ... - Compact GL383527.1 ... - Compact GL383528.1 ... - Compact GL383530.1 ... - Compact GL383531.1 ... - Compact GL383532.1 ... - Compact GL383533.1 ... - Compact GL383534.2 ... - Compact GL383539.1 ... - Compact GL383540.1 ... - Compact GL383541.1 ... - Compact GL383542.1 ... - Compact GL383545.1 ... - Compact GL383546.1 ... - Compact GL383547.1 ... - Compact GL383549.1 ... - Compact GL383550.2 ... - Compact GL383551.1 ... - Compact GL383552.1 ... - Compact GL383553.2 ... - Compact GL383554.1 ... - Compact GL383555.2 ... - Compact GL383556.1 ... - Compact GL383557.1 ... - Compact GL383563.3 ... - Compact GL383564.2 ... - Compact GL383565.1 ... - Compact GL383566.1 ... - Compact GL383567.1 ... - Compact GL383568.1 ... - Compact GL383569.1 ... - Compact GL383570.1 ... - Compact GL383571.1 ... - Compact GL383572.1 ... - Compact GL383573.1 ... - Compact GL383574.1 ... - Compact GL383575.2 ... - Compact GL383576.1 ... - Compact GL383577.2 ... - Compact GL383578.2 ... - Compact GL383579.2 ... - Compact GL383580.2 ... - Compact GL383581.2 ... - Compact GL383582.2 ... - Compact GL383583.2 ... - Compact GL582966.2 ... - Compact GL877875.1 ... - Compact GL877876.1 ... - Compact GL949742.1 ... - Compact GL949746.1 ... - Compact GL949747.2 ... - Compact GL949748.2 ... - Compact GL949749.2 ... - Compact GL949750.2 ... - Compact GL949751.2 ... - Compact GL949752.1 ... - Compact GL949753.2 ... - Compact JH159136.1 ... - Compact JH159137.1 ... - Compact JH159146.1 ... - Compact JH159147.1 ... - Compact JH159148.1 ... - Compact JH636055.2 ... - Compact KB021644.2 ... - Compact KB663609.1 ... - Compact KI270302.1 ... - Compact KI270303.1 ... - Compact KI270304.1 ... - Compact KI270305.1 ... - Compact KI270310.1 ... - Compact KI270311.1 ... - Compact KI270312.1 ... - Compact KI270315.1 ... - Compact KI270316.1 ... - Compact KI270317.1 ... - Compact KI270320.1 ... - Compact KI270322.1 ... - Compact KI270329.1 ... - Compact KI270330.1 ... - Compact KI270333.1 ... - Compact KI270334.1 ... - Compact KI270335.1 ... - Compact KI270336.1 ... - Compact KI270337.1 ... - Compact KI270338.1 ... - Compact KI270340.1 ... - Compact KI270362.1 ... - Compact KI270363.1 ... - Compact KI270364.1 ... - Compact KI270366.1 ... - Compact KI270371.1 ... - Compact KI270372.1 ... - Compact KI270373.1 ... - Compact KI270374.1 ... - Compact KI270375.1 ... - Compact KI270376.1 ... - Compact KI270378.1 ... - Compact KI270379.1 ... - Compact KI270381.1 ... - Compact KI270382.1 ... - Compact KI270383.1 ... - Compact KI270384.1 ... - Compact KI270385.1 ... - Compact KI270386.1 ... - Compact KI270387.1 ... - Compact KI270388.1 ... - Compact KI270389.1 ... - Compact KI270390.1 ... - Compact KI270391.1 ... - Compact KI270392.1 ... - Compact KI270393.1 ... - Compact KI270394.1 ... - Compact KI270395.1 ... - Compact KI270396.1 ... - Compact KI270411.1 ... - Compact KI270412.1 ... - Compact KI270414.1 ... - Compact KI270417.1 ... - Compact KI270418.1 ... - Compact KI270419.1 ... - Compact KI270420.1 ... - Compact KI270422.1 ... - Compact KI270423.1 ... - Compact KI270424.1 ... - Compact KI270425.1 ... - Compact KI270429.1 ... - Compact KI270435.1 ... - Compact KI270438.1 ... - Compact KI270442.1 ... - Compact KI270448.1 ... - Compact KI270465.1 ... - Compact KI270466.1 ... - Compact KI270467.1 ... - Compact KI270468.1 ... - Compact KI270507.1 ... - Compact KI270508.1 ... - Compact KI270509.1 ... - Compact KI270510.1 ... - Compact KI270511.1 ... - Compact KI270512.1 ... - Compact KI270515.1 ... - Compact KI270516.1 ... - Compact KI270517.1 ... - Compact KI270518.1 ... - Compact KI270519.1 ... - Compact KI270521.1 ... - Compact KI270522.1 ... - Compact KI270528.1 ... - Compact KI270529.1 ... - Compact KI270530.1 ... - Compact KI270538.1 ... - Compact KI270539.1 ... - Compact KI270544.1 ... - Compact KI270548.1 ... - Compact KI270579.1 ... - Compact KI270580.1 ... - Compact KI270581.1 ... - Compact KI270582.1 ... - Compact KI270583.1 ... - Compact KI270584.1 ... - Compact KI270587.1 ... - Compact KI270588.1 ... - Compact KI270589.1 ... - Compact KI270590.1 ... - Compact KI270591.1 ... - Compact KI270593.1 ... - Compact KI270706.1 ... - Compact KI270707.1 ... - Compact KI270708.1 ... - Compact KI270709.1 ... - Compact KI270710.1 ... - Compact KI270711.1 ... - Write KI270711.1 to bigWig ... - Compact KI270712.1 ... - Compact KI270713.1 ... - Write KI270713.1 to bigWig ... - Compact KI270714.1 ... - Compact KI270715.1 ... - Compact KI270716.1 ... - Compact KI270717.1 ... - Compact KI270718.1 ... - Compact KI270719.1 ... - Compact KI270720.1 ... - Compact KI270721.1 ... - Write KI270721.1 to bigWig ... - Compact KI270722.1 ... - Compact KI270723.1 ... - Compact KI270724.1 ... - Compact KI270725.1 ... - Compact KI270726.1 ... - Write KI270726.1 to bigWig ... - Compact KI270727.1 ... - Write KI270727.1 to bigWig ... - Compact KI270728.1 ... - Write KI270728.1 to bigWig ... - Compact KI270729.1 ... - Compact KI270730.1 ... - Compact KI270731.1 ... - Write KI270731.1 to bigWig ... - Compact KI270732.1 ... - Compact KI270733.1 ... - Compact KI270734.1 ... - Write KI270734.1 to bigWig ... - Compact KI270735.1 ... - Compact KI270736.1 ... - Compact KI270737.1 ... - Compact KI270738.1 ... - Compact KI270739.1 ... - Compact KI270740.1 ... - Compact KI270741.1 ... - Compact KI270742.1 ... - Compact KI270743.1 ... - Compact KI270744.1 ... - Compact KI270745.1 ... - Compact KI270746.1 ... - Compact KI270747.1 ... - Compact KI270748.1 ... - Compact KI270749.1 ... - Compact KI270750.1 ... - Compact KI270751.1 ... - Compact KI270752.1 ... - Compact KI270753.1 ... - Compact KI270754.1 ... - Compact KI270755.1 ... - Compact KI270756.1 ... - Compact KI270757.1 ... - Compact KI270758.1 ... - Compact KI270759.1 ... - Compact KI270760.1 ... - Compact KI270761.1 ... Add chromosome sizes to bigWig header Number of fragments: 1544628 Split fragments df by chromosome Calculate depth per chromosome: - GL000194.1 ... - GL000195.1 ... - GL000205.2 ... - GL000219.1 ... - KI270711.1 ... - KI270713.1 ... - KI270721.1 ... - KI270726.1 ... - KI270727.1 ... - KI270731.1 ... - KI270734.1 ... - chr1 ... - chr10 ... - chr11 ... - chr12 ... - chr13 ... - chr14 ... - chr15 ... - chr16 ... - chr17 ... - chr18 ... - chr19 ... - chr2 ... - chr20 ... - chr21 ... - chr22 ... - chr3 ... - chr4 ... - chr5 ... - chr6 ... - chr7 ... - chr8 ... - chr9 ... - chrX ... - chrY ... Compact depth array per chromosome (make ranges for consecutive the same values and remove zeros): - Compact GL000008.2 ... - Compact GL000009.2 ... - Compact GL000194.1 ... - Write GL000194.1 to bigWig ... - Compact GL000195.1 ... - Write GL000195.1 to bigWig ... - Compact GL000205.2 ... - Write GL000205.2 to bigWig ... - Compact GL000208.1 ... - Compact GL000209.2 ... - Compact GL000213.1 ... - Compact GL000214.1 ... - Compact GL000216.2 ... - Compact GL000218.1 ... - Compact GL000219.1 ... - Write GL000219.1 to bigWig ... - Compact GL000220.1 ... - Compact GL000221.1 ... - Compact GL000224.1 ... - Compact GL000225.1 ... - Compact GL000226.1 ... - Compact GL000250.2 ... - Compact GL000251.2 ... - Compact GL000252.2 ... - Compact GL000253.2 ... - Compact GL000254.2 ... - Compact GL000255.2 ... - Compact GL000256.2 ... - Compact GL000257.2 ... - Compact GL000258.2 ... - Compact GL339449.2 ... - Compact GL383518.1 ... - Compact GL383519.1 ... - Compact GL383520.2 ... - Compact GL383521.1 ... - Compact GL383522.1 ... - Compact GL383526.1 ... - Compact GL383527.1 ... - Compact GL383528.1 ... - Compact GL383530.1 ... - Compact GL383531.1 ... - Compact GL383532.1 ... - Compact GL383533.1 ... - Compact GL383534.2 ... - Compact GL383539.1 ... - Compact GL383540.1 ... - Compact GL383541.1 ... - Compact GL383542.1 ... - Compact GL383545.1 ... - Compact GL383546.1 ... - Compact GL383547.1 ... - Compact GL383549.1 ... - Compact GL383550.2 ... - Compact GL383551.1 ... - Compact GL383552.1 ... - Compact GL383553.2 ... - Compact GL383554.1 ... - Compact GL383555.2 ... - Compact GL383556.1 ... - Compact GL383557.1 ... - Compact GL383563.3 ... - Compact GL383564.2 ... - Compact GL383565.1 ... - Compact GL383566.1 ... - Compact GL383567.1 ... - Compact GL383568.1 ... - Compact GL383569.1 ... - Compact GL383570.1 ... - Compact GL383571.1 ... - Compact GL383572.1 ... - Compact GL383573.1 ... - Compact GL383574.1 ... - Compact GL383575.2 ... - Compact GL383576.1 ... - Compact GL383577.2 ... - Compact GL383578.2 ... - Compact GL383579.2 ... - Compact GL383580.2 ... - Compact GL383581.2 ... - Compact GL383582.2 ... - Compact GL383583.2 ... - Compact GL582966.2 ... - Compact GL877875.1 ... - Compact GL877876.1 ... - Compact GL949742.1 ... - Compact GL949746.1 ... - Compact GL949747.2 ... - Compact GL949748.2 ... - Compact GL949749.2 ... - Compact GL949750.2 ... - Compact GL949751.2 ... - Compact GL949752.1 ... - Compact GL949753.2 ... - Compact JH159136.1 ... - Compact JH159137.1 ... - Compact JH159146.1 ... - Compact JH159147.1 ... - Compact JH159148.1 ... - Compact JH636055.2 ... - Compact KB021644.2 ... - Compact KB663609.1 ... - Compact KI270302.1 ... - Compact KI270303.1 ... - Compact KI270304.1 ... - Compact KI270305.1 ... - Compact KI270310.1 ... - Compact KI270311.1 ... - Compact KI270312.1 ... - Compact KI270315.1 ... - Compact KI270316.1 ... - Compact KI270317.1 ... - Compact KI270320.1 ... - Compact KI270322.1 ... - Compact KI270329.1 ... - Compact KI270330.1 ... - Compact KI270333.1 ... - Compact KI270334.1 ... - Compact KI270335.1 ... - Compact KI270336.1 ... - Compact KI270337.1 ... - Compact KI270338.1 ... - Compact KI270340.1 ... - Compact KI270362.1 ... - Compact KI270363.1 ... - Compact KI270364.1 ... - Compact KI270366.1 ... - Compact KI270371.1 ... - Compact KI270372.1 ... - Compact KI270373.1 ... - Compact KI270374.1 ... - Compact KI270375.1 ... - Compact KI270376.1 ... - Compact KI270378.1 ... - Compact KI270379.1 ... - Compact KI270381.1 ... - Compact KI270382.1 ... - Compact KI270383.1 ... - Compact KI270384.1 ... - Compact KI270385.1 ... - Compact KI270386.1 ... - Compact KI270387.1 ... - Compact KI270388.1 ... - Compact KI270389.1 ... - Compact KI270390.1 ... - Compact KI270391.1 ... - Compact KI270392.1 ... - Compact KI270393.1 ... - Compact KI270394.1 ... - Compact KI270395.1 ... - Compact KI270396.1 ... - Compact KI270411.1 ... - Compact KI270412.1 ... - Compact KI270414.1 ... - Compact KI270417.1 ... - Compact KI270418.1 ... - Compact KI270419.1 ... - Compact KI270420.1 ... - Compact KI270422.1 ... - Compact KI270423.1 ... - Compact KI270424.1 ... - Compact KI270425.1 ... - Compact KI270429.1 ... - Compact KI270435.1 ... - Compact KI270438.1 ... - Compact KI270442.1 ... - Compact KI270448.1 ... - Compact KI270465.1 ... - Compact KI270466.1 ... - Compact KI270467.1 ... - Compact KI270468.1 ... - Compact KI270507.1 ... - Compact KI270508.1 ... - Compact KI270509.1 ... - Compact KI270510.1 ... - Compact KI270511.1 ... - Compact KI270512.1 ... - Compact KI270515.1 ... - Compact KI270516.1 ... - Compact KI270517.1 ... - Compact KI270518.1 ... - Compact KI270519.1 ... - Compact KI270521.1 ... - Compact KI270522.1 ... - Compact KI270528.1 ... - Compact KI270529.1 ... - Compact KI270530.1 ... - Compact KI270538.1 ... - Compact KI270539.1 ... - Compact KI270544.1 ... - Compact KI270548.1 ... - Compact KI270579.1 ... - Compact KI270580.1 ... - Compact KI270581.1 ... - Compact KI270582.1 ... - Compact KI270583.1 ... - Compact KI270584.1 ... - Compact KI270587.1 ... - Compact KI270588.1 ... - Compact KI270589.1 ... - Compact KI270590.1 ... - Compact KI270591.1 ... - Compact KI270593.1 ... - Compact KI270706.1 ... - Compact KI270707.1 ... - Compact KI270708.1 ... - Compact KI270709.1 ... - Compact KI270710.1 ... - Compact KI270711.1 ... - Write KI270711.1 to bigWig ... - Compact KI270712.1 ... - Compact KI270713.1 ... - Write KI270713.1 to bigWig ... - Compact KI270714.1 ... - Compact KI270715.1 ... - Compact KI270716.1 ... - Compact KI270717.1 ... - Compact KI270718.1 ... - Compact KI270719.1 ... - Compact KI270720.1 ... - Compact KI270721.1 ... - Write KI270721.1 to bigWig ... - Compact KI270722.1 ... - Compact KI270723.1 ... - Compact KI270724.1 ... - Compact KI270725.1 ... - Compact KI270726.1 ... - Write KI270726.1 to bigWig ... - Compact KI270727.1 ... - Write KI270727.1 to bigWig ... - Compact KI270728.1 ... - Compact KI270729.1 ... - Compact KI270730.1 ... - Compact KI270731.1 ... - Write KI270731.1 to bigWig ... - Compact KI270732.1 ... - Compact KI270733.1 ... - Compact KI270734.1 ... - Write KI270734.1 to bigWig ... - Compact KI270735.1 ... - Compact KI270736.1 ... - Compact KI270737.1 ... - Compact KI270738.1 ... - Compact KI270739.1 ... - Compact KI270740.1 ... - Compact KI270741.1 ... - Compact KI270742.1 ... - Compact KI270743.1 ... - Compact KI270744.1 ... - Compact KI270745.1 ... - Compact KI270746.1 ... - Compact KI270747.1 ... - Compact KI270748.1 ... - Compact KI270749.1 ... - Compact KI270750.1 ... - Compact KI270751.1 ... - Compact KI270752.1 ... - Compact KI270753.1 ... - Compact KI270754.1 ... - Compact KI270755.1 ... - Compact KI270756.1 ... - Compact KI270757.1 ... - Compact KI270758.1 ... - Compact KI270759.1 ... - Compact KI270760.1 ... - Compact KI270761.1 ... - Compact KI270762.1 ... - Compact KI270763.1 ... - Compact KI270764.1 ... - Compact KI270765.1 ... - Compact KI270766.1 ... - Compact KI270767.1 ...

After this, I run the following to proceed with peak calling:

import pickle
bed_paths = pickle.load(open(os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bed_files/bed_paths.pkl'), 'rb'))
bw_paths =  pickle.load(open(os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bed_files/bw_paths.pkl'), 'rb'))
from pycisTopic.pseudobulk_peak_calling import peak_calling
macs_path='macs2'
# Run peak calling
narrow_peaks_dict = peak_calling(macs_path,
                                 bed_paths,
                                 os.path.join(work_dir, 'scATAC/consensus_peak_calling/MACS/'),
                                 genome_size='hs',
                                 n_cpu=1,
                                 input_format='BEDPE',
                                 shift=73, 
                                 ext_size=146,
                                 keep_dup = 'all',
                                 q_value = 0.05,
                                 _temp_dir = os.path.join(tmp_dir, 'ray_spill'))

And as you probably expect, due to strange previous output, now I get this error: ileNotFoundError: [Errno 2] No such file or directory: \'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells_1.bed.gz\'\n'

More detailed error

2024-02-13 21:39:15,305 INFO worker.py:1715 -- Started a local Ray instance. View the dashboard at http://127.0.0.1:8265 (macs_call_peak_ray pid=893658) 2024-02-13 21:39:18,724 cisTopic INFO Calling peaks for FCGR3A_Monocytes with macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/FCGR3A_Monocytes.bed.gz --name FCGR3A_Monocytes --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda (macs_call_peak_ray pid=893655) 2024-02-13 21:39:18,780 cisTopic INFO Calling peaks for Dendritic_cells with macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/Dendritic_cells.bed.gz --name Dendritic_cells --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda (macs_call_peak_ray pid=893661) 2024-02-13 21:39:18,776 cisTopic INFO Calling peaks for B_cells with macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells.bed.gz --name B_cells --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda --------------------------------------------------------------------------- RayTaskError(RuntimeError) Traceback (most recent call last) Cell In[29], line 7 5 macs_path='macs2' 6 # Run peak calling ----> 7 narrow_peaks_dict = peak_calling(macs_path, 8 bed_paths, 9 os.path.join(work_dir, 'scATAC/consensus_peak_calling/MACS/'), 10 genome_size='hs', 11 n_cpu=8, 12 input_format='BEDPE', 13 shift=73, 14 ext_size=146, 15 keep_dup = 'all', 16 q_value = 0.05, 17 _temp_dir = os.path.join(tmp_dir, 'ray_spill')) File ~/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py:383, in peak_calling(macs_path, bed_paths, outdir, genome_size, n_cpu, input_format, shift, ext_size, keep_dup, q_value, nolambda, **kwargs) 380 os.makedirs(outdir) 382 ray.init(num_cpus=n_cpu, **kwargs) --> 383 narrow_peaks = ray.get( 384 [ 385 macs_call_peak_ray.remote( 386 macs_path, 387 bed_paths[name], 388 name, 389 outdir, 390 genome_size, 391 input_format, 392 shift, 393 ext_size, 394 keep_dup, 395 q_value, 396 nolambda) for name in list( 397 bed_paths.keys())]) 398 ray.shutdown() 399 narrow_peaks_dict = { 400 list( 401 bed_paths.keys())[i]: narrow_peaks[i].narrow_peak for i in range( 402 len(narrow_peaks))} File ~/.local/lib/python3.8/site-packages/ray/_private/auto_init_hook.py:22, in wrap_auto_init..auto_init_wrapper(*args, **kwargs) 19 @wraps(fn) 20 def auto_init_wrapper(*args, **kwargs): 21 auto_init_ray() ---> 22 return fn(*args, **kwargs) File ~/.local/lib/python3.8/site-packages/ray/_private/client_mode_hook.py:103, in client_mode_hook..wrapper(*args, **kwargs) 101 if func.__name__ != "init" or is_client_mode_enabled_by_default: 102 return getattr(ray, func.__name__)(*args, **kwargs) --> 103 return func(*args, **kwargs) File ~/.local/lib/python3.8/site-packages/ray/_private/worker.py:2624, in get(object_refs, timeout) 2622 worker.core_worker.dump_object_store_memory_usage() 2623 if isinstance(value, RayTaskError): -> 2624 raise value.as_instanceof_cause() 2625 else: 2626 raise value RayTaskError(RuntimeError): ray::macs_call_peak_ray() (pid=893658, ip=129.79.247.65) File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 415, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/FCGR3A_Monocytes.bed.gz --name FCGR3A_Monocytes --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' returned non-zero exit status 1. During handling of the above exception, another exception occurred: ray::macs_call_peak_ray() (pid=893658, ip=129.79.247.65) File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 459, in macs_call_peak_ray MACS_peak_calling = MACSCallPeak( File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 527, in __init__ self.call_peak() File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 556, in call_peak raise RuntimeError( RuntimeError: command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/FCGR3A_Monocytes.bed.gz --name FCGR3A_Monocytes --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' return with error (code 1): b'INFO @ Tue, 13 Feb 2024 21:39:19: \n# Command line: callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/FCGR3A_Monocytes.bed.gz --name FCGR3A_Monocytes --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda\n# ARGUMENTS LIST:\n# name = FCGR3A_Monocytes\n# format = BEDPE\n# ChIP-seq file = [\'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/FCGR3A_Monocytes.bed.gz\']\n# control file = None\n# effective genome size = 2.70e+09\n# band width = 300\n# model fold = [5, 50]\n# qvalue cutoff = 5.00e-02\n# The maximum gap between significant sites is assigned as the read length/tag size.\n# The minimum length of peaks is assigned as the predicted fragment length "d".\n# Larger dataset will be scaled towards smaller dataset.\n# Range for calculating regional lambda is: 10000 bps\n# Broad region calling is off\n# Paired-End mode is on\n# Searching for subpeak summits is on\n \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read fragment files... \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read treatment fragments... \nTraceback (most recent call last):\n File "/home/ejafari/.local/bin/macs2", line 653, in \n main()\n File "/home/ejafari/.local/bin/macs2", line 51, in main\n run( args )\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 64, in run\n if options.PE_MODE: (treat, control) = load_frag_files_options (options)\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 349, in load_frag_files_options\n tp = options.parser(options.tfile[0], buffer_size=options.buffer_size)\n File "MACS2/IO/Parser.pyx", line 330, in MACS2.IO.Parser.GenericParser.__init__\n File "/usr/lib/python3.8/gzip.py", line 58, in open\n binary_file = GzipFile(filename, gz_mode, compresslevel)\n File "/usr/lib/python3.8/gzip.py", line 173, in __init__\n fileobj = self.myfileobj = builtins.open(filename, mode or \'rb\')\nFileNotFoundError: [Errno 2] No such file or directory: \'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/FCGR3A_Monocytes.bed.gz\'\n' 2024-02-13 21:39:24,808 ERROR worker.py:405 -- Unhandled error (suppress with 'RAY_IGNORE_UNHANDLED_ERRORS=1'): ray::macs_call_peak_ray() (pid=893661, ip=129.79.247.65) File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 415, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells.bed.gz --name B_cells --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' returned non-zero exit status 1. During handling of the above exception, another exception occurred: ray::macs_call_peak_ray() (pid=893661, ip=129.79.247.65) File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 459, in macs_call_peak_ray MACS_peak_calling = MACSCallPeak( File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 527, in __init__ self.call_peak() File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 556, in call_peak raise RuntimeError( RuntimeError: command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells.bed.gz --name B_cells --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' return with error (code 1): b'INFO @ Tue, 13 Feb 2024 21:39:19: \n# Command line: callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells.bed.gz --name B_cells --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda\n# ARGUMENTS LIST:\n# name = B_cells\n# format = BEDPE\n# ChIP-seq file = [\'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells.bed.gz\']\n# control file = None\n# effective genome size = 2.70e+09\n# band width = 300\n# model fold = [5, 50]\n# qvalue cutoff = 5.00e-02\n# The maximum gap between significant sites is assigned as the read length/tag size.\n# The minimum length of peaks is assigned as the predicted fragment length "d".\n# Larger dataset will be scaled towards smaller dataset.\n# Range for calculating regional lambda is: 10000 bps\n# Broad region calling is off\n# Paired-End mode is on\n# Searching for subpeak summits is on\n \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read fragment files... \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read treatment fragments... \nTraceback (most recent call last):\n File "/home/ejafari/.local/bin/macs2", line 653, in \n main()\n File "/home/ejafari/.local/bin/macs2", line 51, in main\n run( args )\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 64, in run\n if options.PE_MODE: (treat, control) = load_frag_files_options (options)\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 349, in load_frag_files_options\n tp = options.parser(options.tfile[0], buffer_size=options.buffer_size)\n File "MACS2/IO/Parser.pyx", line 330, in MACS2.IO.Parser.GenericParser.__init__\n File "/usr/lib/python3.8/gzip.py", line 58, in open\n binary_file = GzipFile(filename, gz_mode, compresslevel)\n File "/usr/lib/python3.8/gzip.py", line 173, in __init__\n fileobj = self.myfileobj = builtins.open(filename, mode or \'rb\')\nFileNotFoundError: [Errno 2] No such file or directory: \'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells.bed.gz\'\n' 2024-02-13 21:39:24,810 ERROR worker.py:405 -- Unhandled error (suppress with 'RAY_IGNORE_UNHANDLED_ERRORS=1'): ray::macs_call_peak_ray() (pid=893660, ip=129.79.247.65) File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 415, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells_2.bed.gz --name B_cells_2 --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' returned non-zero exit status 1. During handling of the above exception, another exception occurred: ray::macs_call_peak_ray() (pid=893660, ip=129.79.247.65) File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 459, in macs_call_peak_ray MACS_peak_calling = MACSCallPeak( File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 527, in __init__ self.call_peak() File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 556, in call_peak raise RuntimeError( RuntimeError: command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells_2.bed.gz --name B_cells_2 --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' return with error (code 1): b'INFO @ Tue, 13 Feb 2024 21:39:19: \n# Command line: callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells_2.bed.gz --name B_cells_2 --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda\n# ARGUMENTS LIST:\n# name = B_cells_2\n# format = BEDPE\n# ChIP-seq file = [\'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells_2.bed.gz\']\n# control file = None\n# effective genome size = 2.70e+09\n# band width = 300\n# model fold = [5, 50]\n# qvalue cutoff = 5.00e-02\n# The maximum gap between significant sites is assigned as the read length/tag size.\n# The minimum length of peaks is assigned as the predicted fragment length "d".\n# Larger dataset will be scaled towards smaller dataset.\n# Range for calculating regional lambda is: 10000 bps\n# Broad region calling is off\n# Paired-End mode is on\n# Searching for subpeak summits is on\n \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read fragment files... \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read treatment fragments... \nTraceback (most recent call last):\n File "/home/ejafari/.local/bin/macs2", line 653, in \n main()\n File "/home/ejafari/.local/bin/macs2", line 51, in main\n run( args )\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 64, in run\n if options.PE_MODE: (treat, control) = load_frag_files_options (options)\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 349, in load_frag_files_options\n tp = options.parser(options.tfile[0], buffer_size=options.buffer_size)\n File "MACS2/IO/Parser.pyx", line 330, in MACS2.IO.Parser.GenericParser.__init__\n File "/usr/lib/python3.8/gzip.py", line 58, in open\n binary_file = GzipFile(filename, gz_mode, compresslevel)\n File "/usr/lib/python3.8/gzip.py", line 173, in __init__\n fileobj = self.myfileobj = builtins.open(filename, mode or \'rb\')\nFileNotFoundError: [Errno 2] No such file or directory: \'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells_2.bed.gz\'\n' 2024-02-13 21:39:24,812 ERROR worker.py:405 -- Unhandled error (suppress with 'RAY_IGNORE_UNHANDLED_ERRORS=1'): ray::macs_call_peak_ray() (pid=893655, ip=129.79.247.65) File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 415, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/Dendritic_cells.bed.gz --name Dendritic_cells --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' returned non-zero exit status 1. During handling of the above exception, another exception occurred: ray::macs_call_peak_ray() (pid=893655, ip=129.79.247.65) File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 459, in macs_call_peak_ray MACS_peak_calling = MACSCallPeak( File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 527, in __init__ self.call_peak() File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 556, in call_peak raise RuntimeError( RuntimeError: command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/Dendritic_cells.bed.gz --name Dendritic_cells --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' return with error (code 1): b'INFO @ Tue, 13 Feb 2024 21:39:19: \n# Command line: callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/Dendritic_cells.bed.gz --name Dendritic_cells --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda\n# ARGUMENTS LIST:\n# name = Dendritic_cells\n# format = BEDPE\n# ChIP-seq file = [\'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/Dendritic_cells.bed.gz\']\n# control file = None\n# effective genome size = 2.70e+09\n# band width = 300\n# model fold = [5, 50]\n# qvalue cutoff = 5.00e-02\n# The maximum gap between significant sites is assigned as the read length/tag size.\n# The minimum length of peaks is assigned as the predicted fragment length "d".\n# Larger dataset will be scaled towards smaller dataset.\n# Range for calculating regional lambda is: 10000 bps\n# Broad region calling is off\n# Paired-End mode is on\n# Searching for subpeak summits is on\n \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read fragment files... \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read treatment fragments... \nTraceback (most recent call last):\n File "/home/ejafari/.local/bin/macs2", line 653, in \n main()\n File "/home/ejafari/.local/bin/macs2", line 51, in main\n run( args )\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 64, in run\n if options.PE_MODE: (treat, control) = load_frag_files_options (options)\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 349, in load_frag_files_options\n tp = options.parser(options.tfile[0], buffer_size=options.buffer_size)\n File "MACS2/IO/Parser.pyx", line 330, in MACS2.IO.Parser.GenericParser.__init__\n File "/usr/lib/python3.8/gzip.py", line 58, in open\n binary_file = GzipFile(filename, gz_mode, compresslevel)\n File "/usr/lib/python3.8/gzip.py", line 173, in __init__\n fileobj = self.myfileobj = builtins.open(filename, mode or \'rb\')\nFileNotFoundError: [Errno 2] No such file or directory: \'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/Dendritic_cells.bed.gz\'\n' 2024-02-13 21:39:24,814 ERROR worker.py:405 -- Unhandled error (suppress with 'RAY_IGNORE_UNHANDLED_ERRORS=1'): ray::macs_call_peak_ray() (pid=893657, ip=129.79.247.65) File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 415, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/CD14_Monocytes.bed.gz --name CD14_Monocytes --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' returned non-zero exit status 1. During handling of the above exception, another exception occurred: ray::macs_call_peak_ray() (pid=893657, ip=129.79.247.65) File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 459, in macs_call_peak_ray MACS_peak_calling = MACSCallPeak( File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 527, in __init__ self.call_peak() File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 556, in call_peak raise RuntimeError( RuntimeError: command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/CD14_Monocytes.bed.gz --name CD14_Monocytes --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' return with error (code 1): b'INFO @ Tue, 13 Feb 2024 21:39:19: \n# Command line: callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/CD14_Monocytes.bed.gz --name CD14_Monocytes --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda\n# ARGUMENTS LIST:\n# name = CD14_Monocytes\n# format = BEDPE\n# ChIP-seq file = [\'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/CD14_Monocytes.bed.gz\']\n# control file = None\n# effective genome size = 2.70e+09\n# band width = 300\n# model fold = [5, 50]\n# qvalue cutoff = 5.00e-02\n# The maximum gap between significant sites is assigned as the read length/tag size.\n# The minimum length of peaks is assigned as the predicted fragment length "d".\n# Larger dataset will be scaled towards smaller dataset.\n# Range for calculating regional lambda is: 10000 bps\n# Broad region calling is off\n# Paired-End mode is on\n# Searching for subpeak summits is on\n \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read fragment files... \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read treatment fragments... \nTraceback (most recent call last):\n File "/home/ejafari/.local/bin/macs2", line 653, in \n main()\n File "/home/ejafari/.local/bin/macs2", line 51, in main\n run( args )\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 64, in run\n if options.PE_MODE: (treat, control) = load_frag_files_options (options)\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 349, in load_frag_files_options\n tp = options.parser(options.tfile[0], buffer_size=options.buffer_size)\n File "MACS2/IO/Parser.pyx", line 330, in MACS2.IO.Parser.GenericParser.__init__\n File "/usr/lib/python3.8/gzip.py", line 58, in open\n binary_file = GzipFile(filename, gz_mode, compresslevel)\n File "/usr/lib/python3.8/gzip.py", line 173, in __init__\n fileobj = self.myfileobj = builtins.open(filename, mode or \'rb\')\nFileNotFoundError: [Errno 2] No such file or directory: \'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/CD14_Monocytes.bed.gz\'\n' 2024-02-13 21:39:24,815 ERROR worker.py:405 -- Unhandled error (suppress with 'RAY_IGNORE_UNHANDLED_ERRORS=1'): ray::macs_call_peak_ray() (pid=893654, ip=129.79.247.65) File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 415, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/CD4_T_cells.bed.gz --name CD4_T_cells --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' returned non-zero exit status 1. During handling of the above exception, another exception occurred: ray::macs_call_peak_ray() (pid=893654, ip=129.79.247.65) File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 459, in macs_call_peak_ray MACS_peak_calling = MACSCallPeak( File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 527, in __init__ self.call_peak() File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 556, in call_peak raise RuntimeError( RuntimeError: command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/CD4_T_cells.bed.gz --name CD4_T_cells --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' return with error (code 1): b'INFO @ Tue, 13 Feb 2024 21:39:19: \n# Command line: callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/CD4_T_cells.bed.gz --name CD4_T_cells --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda\n# ARGUMENTS LIST:\n# name = CD4_T_cells\n# format = BEDPE\n# ChIP-seq file = [\'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/CD4_T_cells.bed.gz\']\n# control file = None\n# effective genome size = 2.70e+09\n# band width = 300\n# model fold = [5, 50]\n# qvalue cutoff = 5.00e-02\n# The maximum gap between significant sites is assigned as the read length/tag size.\n# The minimum length of peaks is assigned as the predicted fragment length "d".\n# Larger dataset will be scaled towards smaller dataset.\n# Range for calculating regional lambda is: 10000 bps\n# Broad region calling is off\n# Paired-End mode is on\n# Searching for subpeak summits is on\n \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read fragment files... \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read treatment fragments... \nTraceback (most recent call last):\n File "/home/ejafari/.local/bin/macs2", line 653, in \n main()\n File "/home/ejafari/.local/bin/macs2", line 51, in main\n run( args )\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 64, in run\n if options.PE_MODE: (treat, control) = load_frag_files_options (options)\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 349, in load_frag_files_options\n tp = options.parser(options.tfile[0], buffer_size=options.buffer_size)\n File "MACS2/IO/Parser.pyx", line 330, in MACS2.IO.Parser.GenericParser.__init__\n File "/usr/lib/python3.8/gzip.py", line 58, in open\n binary_file = GzipFile(filename, gz_mode, compresslevel)\n File "/usr/lib/python3.8/gzip.py", line 173, in __init__\n fileobj = self.myfileobj = builtins.open(filename, mode or \'rb\')\nFileNotFoundError: [Errno 2] No such file or directory: \'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/CD4_T_cells.bed.gz\'\n' 2024-02-13 21:39:24,816 ERROR worker.py:405 -- Unhandled error (suppress with 'RAY_IGNORE_UNHANDLED_ERRORS=1'): ray::macs_call_peak_ray() (pid=893656, ip=129.79.247.65) File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 415, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/CD8_T_cells.bed.gz --name CD8_T_cells --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' returned non-zero exit status 1. During handling of the above exception, another exception occurred: ray::macs_call_peak_ray() (pid=893656, ip=129.79.247.65) File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 459, in macs_call_peak_ray MACS_peak_calling = MACSCallPeak( File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 527, in __init__ self.call_peak() File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 556, in call_peak raise RuntimeError( RuntimeError: command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/CD8_T_cells.bed.gz --name CD8_T_cells --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' return with error (code 1): b'INFO @ Tue, 13 Feb 2024 21:39:19: \n# Command line: callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/CD8_T_cells.bed.gz --name CD8_T_cells --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda\n# ARGUMENTS LIST:\n# name = CD8_T_cells\n# format = BEDPE\n# ChIP-seq file = [\'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/CD8_T_cells.bed.gz\']\n# control file = None\n# effective genome size = 2.70e+09\n# band width = 300\n# model fold = [5, 50]\n# qvalue cutoff = 5.00e-02\n# The maximum gap between significant sites is assigned as the read length/tag size.\n# The minimum length of peaks is assigned as the predicted fragment length "d".\n# Larger dataset will be scaled towards smaller dataset.\n# Range for calculating regional lambda is: 10000 bps\n# Broad region calling is off\n# Paired-End mode is on\n# Searching for subpeak summits is on\n \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read fragment files... \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read treatment fragments... \nTraceback (most recent call last):\n File "/home/ejafari/.local/bin/macs2", line 653, in \n main()\n File "/home/ejafari/.local/bin/macs2", line 51, in main\n run( args )\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 64, in run\n if options.PE_MODE: (treat, control) = load_frag_files_options (options)\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 349, in load_frag_files_options\n tp = options.parser(options.tfile[0], buffer_size=options.buffer_size)\n File "MACS2/IO/Parser.pyx", line 330, in MACS2.IO.Parser.GenericParser.__init__\n File "/usr/lib/python3.8/gzip.py", line 58, in open\n binary_file = GzipFile(filename, gz_mode, compresslevel)\n File "/usr/lib/python3.8/gzip.py", line 173, in __init__\n fileobj = self.myfileobj = builtins.open(filename, mode or \'rb\')\nFileNotFoundError: [Errno 2] No such file or directory: \'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/CD8_T_cells.bed.gz\'\n' 2024-02-13 21:39:24,817 ERROR worker.py:405 -- Unhandled error (suppress with 'RAY_IGNORE_UNHANDLED_ERRORS=1'): ray::macs_call_peak_ray() (pid=893659, ip=129.79.247.65) File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 415, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/home/ejafari/.conda/envs/scenicplus/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells_1.bed.gz --name B_cells_1 --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' returned non-zero exit status 1. During handling of the above exception, another exception occurred: ray::macs_call_peak_ray() (pid=893659, ip=129.79.247.65) File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 459, in macs_call_peak_ray MACS_peak_calling = MACSCallPeak( File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 527, in __init__ self.call_peak() File "/home/ejafari/.local/lib/python3.8/site-packages/pycisTopic/pseudobulk_peak_calling.py", line 556, in call_peak raise RuntimeError( RuntimeError: command 'macs2 callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells_1.bed.gz --name B_cells_1 --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda' return with error (code 1): b'INFO @ Tue, 13 Feb 2024 21:39:19: \n# Command line: callpeak --treatment pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells_1.bed.gz --name B_cells_1 --outdir pbmc_tutorial/scATAC/consensus_peak_calling/MACS/ --format BEDPE --gsize hs --qvalue 0.05 --nomodel --shift 73 --extsize 146 --keep-dup all --call-summits --nolambda\n# ARGUMENTS LIST:\n# name = B_cells_1\n# format = BEDPE\n# ChIP-seq file = [\'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells_1.bed.gz\']\n# control file = None\n# effective genome size = 2.70e+09\n# band width = 300\n# model fold = [5, 50]\n# qvalue cutoff = 5.00e-02\n# The maximum gap between significant sites is assigned as the read length/tag size.\n# The minimum length of peaks is assigned as the predicted fragment length "d".\n# Larger dataset will be scaled towards smaller dataset.\n# Range for calculating regional lambda is: 10000 bps\n# Broad region calling is off\n# Paired-End mode is on\n# Searching for subpeak summits is on\n \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read fragment files... \nINFO @ Tue, 13 Feb 2024 21:39:19: #1 read treatment fragments... \nTraceback (most recent call last):\n File "/home/ejafari/.local/bin/macs2", line 653, in \n main()\n File "/home/ejafari/.local/bin/macs2", line 51, in main\n run( args )\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 64, in run\n if options.PE_MODE: (treat, control) = load_frag_files_options (options)\n File "/home/ejafari/.local/lib/python3.8/site-packages/MACS2/callpeak_cmd.py", line 349, in load_frag_files_options\n tp = options.parser(options.tfile[0], buffer_size=options.buffer_size)\n File "MACS2/IO/Parser.pyx", line 330, in MACS2.IO.Parser.GenericParser.__init__\n File "/usr/lib/python3.8/gzip.py", line 58, in open\n binary_file = GzipFile(filename, gz_mode, compresslevel)\n File "/usr/lib/python3.8/gzip.py", line 173, in __init__\n fileobj = self.myfileobj = builtins.open(filename, mode or \'rb\')\nFileNotFoundError: [Errno 2] No such file or directory: \'pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/B_cells_1.bed.gz\'\n'

ls pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/

Details

B_cells_1.fragments.tsv.gz CD14+_Monocytes.fragments.tsv.gz B_cells_2.fragments.tsv.gz CD4_T_cells.fragments.tsv.gz B_cells.fragments.tsv.gz CD8_T_cells.fragments.tsv.gz bed_paths.pkl Dendritic_cells.fragments.tsv.gz bw_paths.pkl FCGR3A+_Monocytes.fragments.tsv.gz - Compact KI270772.1 ... - Compact KI270773.1 ... - Compact KI270774.1 ... - Compact KI270775.1 ... - Compact KI270776.1 ... - Compact KI270777.1 ... - Compact KI270778.1 ... - Compact KI270779.1 ... - Compact KI270780.1 ... - Compact KI270781.1 ... - Compact KI270782.1 ... - Compact KI270783.1 ... - Compact KI270784.1 ... - Compact KI270785.1 ... - Compact KI270786.1 ... - Compact KI270787.1 ... - Compact KI270788.1 ... - Compact KI270789.1 ... - Compact KI270790.1 ... - Compact KI270791.1 ... - Compact KI270792.1 ... - Compact KI270793.1 ... - Compact KI270794.1 ... - Compact KI270795.1 ... - Compact KI270796.1 ... - Compact KI270797.1 ... - Compact KI270798.1 ... - Compact KI270799.1 ... - Compact KI270800.1 ... - Compact KI270801.1 ... - Compact KI270802.1 ... - Compact KI270803.1 ... - Compact KI270804.1 ... - Compact KI270805.1 ... - Compact KI270806.1 ... - Compact KI270807.1 ... - Compact KI270808.1 ... - Compact KI270809.1 ... - Compact KI270810.1 ... - Compact KI270811.1 ... - Compact KI270812.1 ... - Compact KI270813.1 ... - Compact KI270814.1 ... - Compact KI270815.1 ... - Compact KI270816.1 ... - Compact KI270817.1 ... - Compact KI270818.1 ... - Compact KI270819.1 ... - Compact KI270820.1 ... - Compact KI270821.1 ... - Compact KI270822.1 ... - Compact KI270823.1 ... - Compact KI270824.1 ... - Compact KI270825.1 ... - Compact KI270826.1 ... - Compact KI270827.1 ... - Compact KI270829.1 ... - Compact KI270830.1 ... - Compact KI270831.1 ... - Compact KI270832.1 ... - Compact KI270833.1 ... - Compact KI270834.1 ... - Compact KI270835.1 ... - Compact KI270836.1 ... - Compact KI270837.1 ... - Compact KI270838.1 ... - Compact KI270839.1 ... - Compact KI270840.1 ... - Compact KI270841.1 ... - Compact KI270842.1 ... - Compact KI270843.1 ... - Compact KI270844.1 ... - Compact KI270845.1 ... - Compact KI270846.1 ... - Compact KI270847.1 ... - Compact KI270848.1 ... - Compact KI270849.1 ... - Compact KI270850.1 ... - Compact KI270851.1 ... - Compact KI270852.1 ... - Compact KI270853.1 ... - Compact KI270854.1 ... - Compact KI270855.1 ... - Compact KI270856.1 ... - Compact KI270857.1 ... - Compact KI270858.1 ... - Compact KI270859.1 ... - Compact KI270860.1 ... - Compact KI270861.1 ... - Compact KI270862.1 ... - Compact KI270863.1 ... - Compact KI270864.1 ... - Compact KI270865.1 ... - Compact KI270866.1 ... - Compact KI270867.1 ... - Compact KI270868.1 ... - Compact KI270869.1 ... - Compact KI270870.1 ... - Compact KI270871.1 ... - Compact KI270872.1 ... - Compact KI270873.1 ... - Compact KI270874.1 ... - Compact KI270875.1 ... - Compact KI270876.1 ... - Compact KI270877.1 ... - Compact KI270878.1 ... - Compact KI270879.1 ... - Compact KI270880.1 ... - Compact KI270881.1 ... - Compact KI270882.1 ... - Compact KI270883.1 ... - Compact KI270884.1 ... - Compact KI270885.1 ... - Compact KI270886.1 ... - Compact KI270887.1 ... - Compact KI270888.1 ... - Compact KI270889.1 ... - Compact KI270890.1 ... - Compact KI270891.1 ... - Compact KI270892.1 ... - Compact KI270893.1 ... - Compact KI270894.1 ... - Compact KI270895.1 ... - Compact KI270896.1 ... - Compact KI270897.1 ... - Compact KI270898.1 ... - Compact KI270899.1 ... - Compact KI270900.1 ... - Compact KI270901.1 ... - Compact KI270902.1 ... - Compact KI270903.1 ... - Compact KI270904.1 ... - Compact KI270905.1 ... - Compact KI270906.1 ... - Compact KI270907.1 ... - Compact KI270908.1 ... - Compact KI270909.1 ... - Compact KI270910.1 ... - Compact KI270911.1 ... - Compact KI270912.1 ... - Compact KI270913.1 ... - Compact KI270914.1 ... - Compact KI270915.1 ... - Compact KI270916.1 ... - Compact KI270917.1 ... - Compact KI270918.1 ... - Compact KI270919.1 ... - Compact KI270920.1 ... - Compact KI270921.1 ... - Compact KI270922.1 ... - Compact KI270923.1 ... - Compact KI270924.1 ... - Compact KI270925.1 ... - Compact KI270926.1 ... - Compact KI270927.1 ... - Compact KI270928.1 ... - Compact KI270929.1 ... - Compact KI270930.1 ... - Compact KI270931.1 ... - Compact KI270932.1 ... - Compact KI270933.1 ... - Compact KI270934.1 ... - Compact KI270935.1 ... - Compact KI270936.1 ... - Compact KI270937.1 ... - Compact KI270938.1 ... - Compact chr1 ... - Write chr1 to bigWig ... - Compact chr2 ... - Write chr2 to bigWig ... - Compact chr3 ... - Write chr3 to bigWig ... - Compact chr4 ... - Write chr4 to bigWig ... - Compact chr5 ... - Write chr5 to bigWig ... - Compact chr6 ... - Write chr6 to bigWig ... - Compact chr7 ... - Write chr7 to bigWig ... - Compact chr8 ... - Write chr8 to bigWig ... - Compact chr9 ... - Write chr9 to bigWig ... - Compact chr10 ... - Write chr10 to bigWig ... - Compact chr11 ... - Write chr11 to bigWig ... - Compact chr12 ... - Write chr12 to bigWig ... - Compact chr13 ... - Write chr13 to bigWig ... - Compact chr14 ... - Write chr14 to bigWig ... - Compact chr15 ... - Write chr15 to bigWig ... - Compact chr16 ... - Write chr16 to bigWig ... - Compact chr17 ... - Write chr17 to bigWig ... - Compact chr18 ... - Write chr18 to bigWig ... - Compact chr19 ... - Write chr19 to bigWig ... - Compact chr20 ... - Write chr20 to bigWig ... - Compact chr21 ... - Write chr21 to bigWig ... - Compact chr22 ... - Write chr22 to bigWig ... - Compact chrM ... - Compact chrX ... - Write chrX to bigWig ... - Compact chrY ... - Write chrY to bigWig ... - Compact KI270774.1 ... - Compact KI270775.1 ... - Compact KI270776.1 ... - Compact KI270777.1 ... - Compact KI270778.1 ... - Compact KI270779.1 ... - Compact KI270780.1 ... - Compact KI270781.1 ... - Compact KI270782.1 ... - Compact KI270783.1 ... - Compact KI270784.1 ... - Compact KI270785.1 ... - Compact KI270786.1 ... - Compact KI270787.1 ... - Compact KI270788.1 ... - Compact KI270789.1 ... - Compact KI270790.1 ... - Compact KI270791.1 ... - Compact KI270792.1 ... - Compact KI270793.1 ... - Compact KI270794.1 ... - Compact KI270795.1 ... - Compact KI270796.1 ... - Compact KI270797.1 ... - Compact KI270798.1 ... - Compact KI270799.1 ... - Compact KI270800.1 ... - Compact KI270801.1 ... - Compact KI270802.1 ... - Compact KI270803.1 ... - Compact KI270804.1 ... - Compact KI270805.1 ... - Compact KI270806.1 ... - Compact KI270807.1 ... - Compact KI270808.1 ... - Compact KI270809.1 ... - Compact KI270810.1 ... - Compact KI270811.1 ... - Compact KI270812.1 ... - Compact KI270813.1 ... - Compact KI270814.1 ... - Compact KI270815.1 ... - Compact KI270816.1 ... - Compact KI270817.1 ... - Compact KI270818.1 ... - Compact KI270819.1 ... - Compact KI270820.1 ... - Compact KI270821.1 ... - Compact KI270822.1 ... - Compact KI270823.1 ... - Compact KI270824.1 ... - Compact KI270825.1 ... - Compact KI270826.1 ... - Compact KI270827.1 ... - Compact KI270829.1 ... - Compact KI270830.1 ... - Compact KI270831.1 ... - Compact KI270832.1 ... - Compact KI270833.1 ... - Compact KI270834.1 ... - Compact KI270835.1 ... - Compact KI270836.1 ... - Compact KI270837.1 ... - Compact KI270838.1 ... - Compact KI270839.1 ... - Compact KI270840.1 ... - Compact KI270841.1 ... - Compact KI270842.1 ... - Compact KI270843.1 ... - Compact KI270844.1 ... - Compact KI270845.1 ... - Compact KI270846.1 ... - Compact KI270847.1 ... - Compact KI270848.1 ... - Compact KI270849.1 ... - Compact KI270850.1 ... - Compact KI270851.1 ... - Compact KI270852.1 ... - Compact KI270853.1 ... - Compact KI270854.1 ... - Compact KI270855.1 ... - Compact KI270856.1 ... - Compact KI270857.1 ... - Compact KI270858.1 ... - Compact KI270859.1 ... - Compact KI270860.1 ... - Compact KI270861.1 ... - Compact KI270862.1 ... - Compact KI270863.1 ... - Compact KI270864.1 ... - Compact KI270865.1 ... - Compact KI270866.1 ... - Compact KI270867.1 ... - Compact KI270868.1 ... - Compact KI270869.1 ... - Compact KI270870.1 ... - Compact KI270871.1 ... - Compact KI270872.1 ... - Compact KI270873.1 ... - Compact KI270874.1 ... - Compact KI270875.1 ... - Compact KI270876.1 ... - Compact KI270877.1 ... - Compact KI270878.1 ... - Compact KI270879.1 ... - Compact KI270880.1 ... - Compact KI270881.1 ... - Compact KI270882.1 ... - Compact KI270883.1 ... - Compact KI270884.1 ... - Compact KI270885.1 ... - Compact KI270886.1 ... - Compact KI270887.1 ... - Compact KI270888.1 ... - Compact KI270889.1 ... - Compact KI270890.1 ... - Compact KI270891.1 ... - Compact KI270892.1 ... - Compact KI270893.1 ... - Compact KI270894.1 ... - Compact KI270895.1 ... - Compact KI270896.1 ... - Compact KI270897.1 ... - Compact KI270898.1 ... - Compact KI270899.1 ... - Compact KI270900.1 ... - Compact KI270901.1 ... - Compact KI270902.1 ... - Compact KI270903.1 ... - Compact KI270904.1 ... - Compact KI270905.1 ... - Compact KI270906.1 ... - Compact KI270907.1 ... - Compact KI270908.1 ... - Compact KI270909.1 ... - Compact KI270910.1 ... - Compact KI270911.1 ... - Compact KI270912.1 ... - Compact KI270913.1 ... - Compact KI270914.1 ... - Compact KI270915.1 ... - Compact KI270916.1 ... - Compact KI270917.1 ... - Compact KI270918.1 ... - Compact KI270919.1 ... - Compact KI270920.1 ... - Compact KI270921.1 ... - Compact KI270922.1 ... - Compact KI270923.1 ... - Compact KI270924.1 ... - Compact KI270925.1 ... - Compact KI270926.1 ... - Compact KI270927.1 ... - Compact KI270928.1 ... - Compact KI270929.1 ... - Compact KI270930.1 ... - Compact KI270931.1 ... - Compact KI270932.1 ... - Compact KI270933.1 ... - Compact KI270934.1 ... - Compact KI270935.1 ... - Compact KI270936.1 ... - Compact KI270937.1 ... - Compact KI270938.1 ... - Compact chr1 ... - Write chr1 to bigWig ... - Compact chr2 ... - Write chr2 to bigWig ... - Compact chr3 ... - Write chr3 to bigWig ... - Compact chr4 ... - Write chr4 to bigWig ... - Compact chr5 ... - Write chr5 to bigWig ... - Compact chr6 ... - Write chr6 to bigWig ... - Compact chr7 ... - Write chr7 to bigWig ... - Compact chr8 ... - Write chr8 to bigWig ... - Compact chr9 ... - Write chr9 to bigWig ... - Compact chr10 ... - Write chr10 to bigWig ... - Compact chr11 ... - Write chr11 to bigWig ... - Compact chr12 ... - Write chr12 to bigWig ... - Compact chr13 ... - Write chr13 to bigWig ... - Compact chr14 ... - Write chr14 to bigWig ... - Compact chr15 ... - Write chr15 to bigWig ... - Compact chr16 ... - Write chr16 to bigWig ... - Compact chr17 ... - Write chr17 to bigWig ... - Compact chr18 ... - Write chr18 to bigWig ... - Compact chr19 ... - Write chr19 to bigWig ... - Compact chr20 ... - Write chr20 to bigWig ... - Compact chr21 ... - Write chr21 to bigWig ... - Compact chr22 ... - Write chr22 to bigWig ... - Compact chrM ... - Compact chrX ... - Write chrX to bigWig ... - Compact chrY ... - Write chrY to bigWig ... - Compact KI270770.1 ... - Compact KI270771.1 ... - Compact KI270772.1 ... - Compact KI270773.1 ... - Compact KI270774.1 ... - Compact KI270775.1 ... - Compact KI270776.1 ... - Compact KI270777.1 ... - Compact KI270778.1 ... - Compact KI270779.1 ... - Compact KI270780.1 ... - Compact KI270781.1 ... - Compact KI270782.1 ... - Compact KI270783.1 ... - Compact KI270784.1 ... - Compact KI270785.1 ... - Compact KI270786.1 ... - Compact KI270787.1 ... - Compact KI270788.1 ... - Compact KI270789.1 ... - Compact KI270790.1 ... - Compact KI270791.1 ... - Compact KI270792.1 ... - Compact KI270793.1 ... - Compact KI270794.1 ... - Compact KI270795.1 ... - Compact KI270796.1 ... - Compact KI270797.1 ... - Compact KI270798.1 ... - Compact KI270799.1 ... - Compact KI270800.1 ... - Compact KI270801.1 ... - Compact KI270802.1 ... - Compact KI270803.1 ... - Compact KI270804.1 ... - Compact KI270805.1 ... - Compact KI270806.1 ... - Compact KI270807.1 ... - Compact KI270808.1 ... - Compact KI270809.1 ... - Compact KI270810.1 ... - Compact KI270811.1 ... - Compact KI270812.1 ... - Compact KI270813.1 ... - Compact KI270814.1 ... - Compact KI270815.1 ... - Compact KI270816.1 ... - Compact KI270817.1 ... - Compact KI270818.1 ... - Compact KI270819.1 ... - Compact KI270820.1 ... - Compact KI270821.1 ... - Compact KI270822.1 ... - Compact KI270823.1 ... - Compact KI270824.1 ... - Compact KI270825.1 ... - Compact KI270826.1 ... - Compact KI270827.1 ... - Compact KI270829.1 ... - Compact KI270830.1 ... - Compact KI270831.1 ... - Compact KI270832.1 ... - Compact KI270833.1 ... - Compact KI270834.1 ... - Compact KI270835.1 ... - Compact KI270836.1 ... - Compact KI270837.1 ... - Compact KI270838.1 ... - Compact KI270839.1 ... - Compact KI270840.1 ... - Compact KI270841.1 ... - Compact KI270842.1 ... - Compact KI270843.1 ... - Compact KI270844.1 ... - Compact KI270845.1 ... - Compact KI270846.1 ... - Compact KI270847.1 ... - Compact KI270848.1 ... - Compact KI270849.1 ... - Compact KI270850.1 ... - Compact KI270851.1 ... - Compact KI270852.1 ... - Compact KI270853.1 ... - Compact KI270854.1 ... - Compact KI270855.1 ... - Compact KI270856.1 ... - Compact KI270857.1 ... - Compact KI270858.1 ... - Compact KI270859.1 ... - Compact KI270860.1 ... - Compact KI270861.1 ... - Compact KI270862.1 ... - Compact KI270863.1 ... - Compact KI270864.1 ... - Compact KI270865.1 ... - Compact KI270866.1 ... - Compact KI270867.1 ... - Compact KI270868.1 ... - Compact KI270869.1 ... - Compact KI270870.1 ... - Compact KI270871.1 ... - Compact KI270872.1 ... - Compact KI270873.1 ... - Compact KI270874.1 ... - Compact KI270875.1 ... - Compact KI270876.1 ... - Compact KI270877.1 ... - Compact KI270878.1 ... - Compact KI270879.1 ... - Compact KI270880.1 ... - Compact KI270881.1 ... - Compact KI270882.1 ... - Compact KI270883.1 ... - Compact KI270884.1 ... - Compact KI270885.1 ... - Compact KI270886.1 ... - Compact KI270887.1 ... - Compact KI270888.1 ... - Compact KI270889.1 ... - Compact KI270890.1 ... - Compact KI270891.1 ... - Compact KI270892.1 ... - Compact KI270893.1 ... - Compact KI270894.1 ... - Compact KI270895.1 ... - Compact KI270896.1 ... - Compact KI270897.1 ... - Compact KI270898.1 ... - Compact KI270899.1 ... - Compact KI270900.1 ... - Compact KI270901.1 ... - Compact KI270902.1 ... - Compact KI270903.1 ... - Compact KI270904.1 ... - Compact KI270905.1 ... - Compact KI270906.1 ... - Compact KI270907.1 ... - Compact KI270908.1 ... - Compact KI270909.1 ... - Compact KI270910.1 ... - Compact KI270911.1 ... - Compact KI270912.1 ... - Compact KI270913.1 ... - Compact KI270914.1 ... - Compact KI270915.1 ... - Compact KI270916.1 ... - Compact KI270917.1 ... - Compact KI270918.1 ... - Compact KI270919.1 ... - Compact KI270920.1 ... - Compact KI270921.1 ... - Compact KI270922.1 ... - Compact KI270923.1 ... - Compact KI270924.1 ... - Compact KI270925.1 ... - Compact KI270926.1 ... - Compact KI270927.1 ... - Compact KI270928.1 ... - Compact KI270929.1 ... - Compact KI270930.1 ... - Compact KI270931.1 ... - Compact KI270932.1 ... - Compact KI270933.1 ... - Compact KI270934.1 ... - Compact KI270935.1 ... - Compact KI270936.1 ... - Compact KI270937.1 ... - Compact KI270938.1 ... - Compact chr1 ... - Write chr1 to bigWig ... - Compact chr2 ... - Write chr2 to bigWig ... - Compact chr3 ... - Write chr3 to bigWig ... - Compact chr4 ... - Write chr4 to bigWig ... - Compact chr5 ... - Write chr5 to bigWig ... - Compact chr6 ... - Write chr6 to bigWig ... - Compact chr7 ... - Write chr7 to bigWig ... - Compact chr8 ... - Write chr8 to bigWig ... - Compact chr9 ... - Write chr9 to bigWig ... - Compact chr10 ... - Write chr10 to bigWig ... - Compact chr11 ... - Write chr11 to bigWig ... - Compact chr12 ... - Write chr12 to bigWig ... - Compact chr13 ... - Write chr13 to bigWig ... - Compact chr14 ... - Write chr14 to bigWig ... - Compact chr15 ... - Write chr15 to bigWig ... - Compact chr16 ... - Write chr16 to bigWig ... - Compact chr17 ... - Write chr17 to bigWig ... - Compact chr18 ... - Write chr18 to bigWig ... - Compact chr19 ... - Write chr19 to bigWig ... - Compact chr20 ... - Write chr20 to bigWig ... - Compact chr21 ... - Write chr21 to bigWig ... - Compact chr22 ... - Write chr22 to bigWig ... - Compact chrM ... - Compact chrX ... - Write chrX to bigWig ... - Compact chrY ... - Write chrY to bigWig ... - Compact KI270768.1 ... - Compact KI270769.1 ... - Compact KI270770.1 ... - Compact KI270771.1 ... - Compact KI270772.1 ... - Compact KI270773.1 ... - Compact KI270774.1 ... - Compact KI270775.1 ... - Compact KI270776.1 ... - Compact KI270777.1 ... - Compact KI270778.1 ... - Compact KI270779.1 ... - Compact KI270780.1 ... - Compact KI270781.1 ... - Compact KI270782.1 ... - Compact KI270783.1 ... - Compact KI270784.1 ... - Compact KI270785.1 ... - Compact KI270786.1 ... - Compact KI270787.1 ... - Compact KI270788.1 ... - Compact KI270789.1 ... - Compact KI270790.1 ... - Compact KI270791.1 ... - Compact KI270792.1 ... - Compact KI270793.1 ... - Compact KI270794.1 ... - Compact KI270795.1 ... - Compact KI270796.1 ... - Compact KI270797.1 ... - Compact KI270798.1 ... - Compact KI270799.1 ... - Compact KI270800.1 ... - Compact KI270801.1 ... - Compact KI270802.1 ... - Compact KI270803.1 ... - Compact KI270804.1 ... - Compact KI270805.1 ... - Compact KI270806.1 ... - Compact KI270807.1 ... - Compact KI270808.1 ... - Compact KI270809.1 ... - Compact KI270810.1 ... - Compact KI270811.1 ... - Compact KI270812.1 ... - Compact KI270813.1 ... - Compact KI270814.1 ... - Compact KI270815.1 ... - Compact KI270816.1 ... - Compact KI270817.1 ... - Compact KI270818.1 ... - Compact KI270819.1 ... - Compact KI270820.1 ... - Compact KI270821.1 ... - Compact KI270822.1 ... - Compact KI270823.1 ... - Compact KI270824.1 ... - Compact KI270825.1 ... - Compact KI270826.1 ... - Compact KI270827.1 ... - Compact KI270829.1 ... - Compact KI270830.1 ... - Compact KI270831.1 ... - Compact KI270832.1 ... - Compact KI270833.1 ... - Compact KI270834.1 ... - Compact KI270835.1 ... - Compact KI270836.1 ... - Compact KI270837.1 ... - Compact KI270838.1 ... - Compact KI270839.1 ... - Compact KI270840.1 ... - Compact KI270841.1 ... - Compact KI270842.1 ... - Compact KI270843.1 ... - Compact KI270844.1 ... - Compact KI270845.1 ... - Compact KI270846.1 ... - Compact KI270847.1 ... - Compact KI270848.1 ... - Compact KI270849.1 ... - Compact KI270850.1 ... - Compact KI270851.1 ... - Compact KI270852.1 ... - Compact KI270853.1 ... - Compact KI270854.1 ... - Compact KI270855.1 ... - Compact KI270856.1 ... - Compact KI270857.1 ... - Compact KI270858.1 ... - Compact KI270859.1 ... - Compact KI270860.1 ... - Compact KI270861.1 ... - Compact KI270862.1 ... - Compact KI270863.1 ... - Compact KI270864.1 ... - Compact KI270865.1 ... - Compact KI270866.1 ... - Compact KI270867.1 ... - Compact KI270868.1 ... - Compact KI270869.1 ... - Compact KI270870.1 ... - Compact KI270871.1 ... - Compact KI270872.1 ... - Compact KI270873.1 ... - Compact KI270874.1 ... - Compact KI270875.1 ... - Compact KI270876.1 ... - Compact KI270877.1 ... - Compact KI270878.1 ... - Compact KI270879.1 ... - Compact KI270880.1 ... - Compact KI270881.1 ... - Compact KI270882.1 ... - Compact KI270883.1 ... - Compact KI270884.1 ... - Compact KI270885.1 ... - Compact KI270886.1 ... - Compact KI270887.1 ... - Compact KI270888.1 ... - Compact KI270889.1 ... - Compact KI270890.1 ... - Compact KI270891.1 ... - Compact KI270892.1 ... - Compact KI270893.1 ... - Compact KI270894.1 ... - Compact KI270895.1 ... - Compact KI270896.1 ... - Compact KI270897.1 ... - Compact KI270898.1 ... - Compact KI270899.1 ... - Compact KI270900.1 ... - Compact KI270901.1 ... - Compact KI270902.1 ... - Compact KI270903.1 ... - Compact KI270904.1 ... - Compact KI270905.1 ... - Compact KI270906.1 ... - Compact KI270907.1 ... - Compact KI270908.1 ... - Compact KI270909.1 ... - Compact KI270910.1 ... - Compact KI270911.1 ... - Compact KI270912.1 ... - Compact KI270913.1 ... - Compact KI270914.1 ... - Compact KI270915.1 ... - Compact KI270916.1 ... - Compact KI270917.1 ... - Compact KI270918.1 ... - Compact KI270919.1 ... - Compact KI270920.1 ... - Compact KI270921.1 ... - Compact KI270922.1 ... - Compact KI270923.1 ... - Compact KI270924.1 ... - Compact KI270925.1 ... - Compact KI270926.1 ... - Compact KI270927.1 ... - Compact KI270928.1 ... - Compact KI270929.1 ... - Compact KI270930.1 ... - Compact KI270931.1 ... - Compact KI270932.1 ... - Compact KI270933.1 ... - Compact KI270934.1 ... - Compact KI270935.1 ... - Compact KI270936.1 ... - Compact KI270937.1 ... - Compact KI270938.1 ... - Compact chr1 ... - Write chr1 to bigWig ... - Compact chr2 ... - Write chr2 to bigWig ... - Compact chr3 ... - Write chr3 to bigWig ... - Compact chr4 ... - Write chr4 to bigWig ... - Compact chr5 ... - Write chr5 to bigWig ... - Compact chr6 ... - Write chr6 to bigWig ... - Compact chr7 ... - Write chr7 to bigWig ... - Compact chr8 ... - Write chr8 to bigWig ... - Compact chr9 ... - Write chr9 to bigWig ... - Compact chr10 ... - Write chr10 to bigWig ... - Compact chr11 ... - Write chr11 to bigWig ... - Compact chr12 ... - Write chr12 to bigWig ... - Compact chr13 ... - Write chr13 to bigWig ... - Compact chr14 ... - Write chr14 to bigWig ... - Compact chr15 ... - Write chr15 to bigWig ... - Compact chr16 ... - Write chr16 to bigWig ... - Compact chr17 ... - Write chr17 to bigWig ... - Compact chr18 ... - Write chr18 to bigWig ... - Compact chr19 ... - Write chr19 to bigWig ... - Compact chr20 ... - Write chr20 to bigWig ... - Compact chr21 ... - Write chr21 to bigWig ... - Compact chr22 ... - Write chr22 to bigWig ... - Compact chrM ... - Compact chrX ... - Write chrX to bigWig ... - Compact chrY ... - Write chrY to bigWig ...

versions:

Could you please let me know if you have any suggestions? Thank you so much!

SeppeDeWinter commented 9 months ago

Hi @wgao688

Thank you for the nice report.

The output you are seeing compact ... is correct, it's true that this does not match anymore with the log messages from the tutorial. So no worries here.

With the following commands:


bed_paths = pickle.load(open(os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bed_files/bed_paths.pkl'), 'rb'))
bw_paths =  pickle.load(open(os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bed_files/bw_paths.pkl'), 'rb'))

are you sure you are not loading the paths of an earlier run. Before the files indeed ended with the extension .bed.gz, now they should end with tsv.gz though.

Can you show the output of:


ls pbmc_tutorial/scATAC/consensus_peak_calling/pseudobulk_bed_files/

All the best,

Seppe

SeppeDeWinter commented 9 months ago

Hi @marck1198

From your error log it does seem like the index file is missing:

See the lines below that I copied from your log.


[E::idx_find_and_load] Could not retrieve index file for 'pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz'
thread '<unnamed>' panicked at src/split_fragments.rs:93:29:
Could not open file pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz

You can either download this file using:


!wget -O pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz.tbi https://cf.10xgenomics.com/samples/cell-arc/2.0.0/pbmc_granulocyte_sorted_3k/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz.tbi

or generate it yourself using


tabix -p bed pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz

Here is a minimal example to reproduce your error.

This is without downloading the index:


wget https://cf.10xgenomics.com/samples/cell-arc/2.0.0/pbmc_granulocyte_sorted_3k/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz
wget https://hgdownload.cse.ucsc.edu/goldenpath/hg38/bigZips/hg38.chrom.sizes

echo -e "sample\tpath_to_fragment_file\nA\tpbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz" > sample_to_fragments.tsv
echo -e "sample\tcell_type\tcell_barcode\nA\ttype_1\tTTTAGCAAGGTAGCTT-1\nA\ttype_1\tATTGACTCAATCCTGA-1\nA\ttype_2\tCCGTTGCGTTAGAGCC-1" > cell_type_to_cell_barcode.tsv

# this function is run under the hood of `export_pseudobulk `
scatac_fragment_tools split \
    --sample_fragments sample_to_fragments.tsv \
    --cell_type_barcodes cell_type_to_cell_barcode.tsv \
    --chrom hg38.chrom.sizes \
    --output .

Produces:


[E::idx_find_and_load] Could not retrieve index file for 'pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz'
thread '<unnamed>' panicked at 'Could not open file pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz', src/split_fragments.rs:93:29
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
Traceback (most recent call last):
  File "/lustre1/project/stg_00002/mambaforge/vsc33053/envs/scatac_fragment_tools/bin/scatac_fragment_tools", line 8, in <module>
    sys.exit(main())
             ^^^^^^
  File "/lustre1/project/stg_00002/mambaforge/vsc33053/envs/scatac_fragment_tools/lib/python3.11/site-packages/scatac_fragment_tools/cli/main.py", line 305, in main
    args.func(args)
  File "/lustre1/project/stg_00002/mambaforge/vsc33053/envs/scatac_fragment_tools/lib/python3.11/site-packages/scatac_fragment_tools/cli/commands.py", line 172, in command_split_fragments_by_cell_type
    split_fragment_files_by_cell_type(
  File "/lustre1/project/stg_00002/mambaforge/vsc33053/envs/scatac_fragment_tools/lib/python3.11/site-packages/scatac_fragment_tools/library/split/split_fragments_by_cell_type.py", line 72, in split_fragment_files_by_cell_type
    joblib.Parallel(n_jobs=n_cpu)(
  File "/lustre1/project/stg_00002/mambaforge/vsc33053/envs/scatac_fragment_tools/lib/python3.11/site-packages/joblib/parallel.py", line 1863, in __call__
    return output if self.return_generator else list(output)
                                                ^^^^^^^^^^^^
  File "/lustre1/project/stg_00002/mambaforge/vsc33053/envs/scatac_fragment_tools/lib/python3.11/site-packages/joblib/parallel.py", line 1792, in _get_sequential_output
    res = func(*args, **kwargs)
          ^^^^^^^^^^^^^^^^^^^^^
pyo3_runtime.PanicException: Could not open file pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz

After downloading the index, everything runs fine


wget https://cf.10xgenomics.com/samples/cell-arc/2.0.0/pbmc_granulocyte_sorted_3k/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz.tbi

scatac_fragment_tools split \
    --sample_fragments sample_to_fragments.tsv \
    --cell_type_barcodes cell_type_to_cell_barcode.tsv \
    --chrom hg38.chrom.sizes \
    --output .

The output looks like this:


❯ ls ./type_*
./type_1.fragments.tsv.gz  ./type_2.fragments.tsv.gz
SeppeDeWinter commented 9 months ago

@laki-spk

It does indeed seem like you do have the index file, so maybe something else is wrong for you.

Can you show the complete output of the function please?

All the best,

Seppe

laki-spk commented 9 months ago

Hi @SeppeDeWinter

Thank you for your help. This is the error message I am getting.

[E::idx_find_and_load] Could not retrieve index file for 'pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz'
thread '<unnamed>' panicked at src/split_fragments.rs:93:29:
Could not open file pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
---------------------------------------------------------------------------
_RemoteTraceback                          Traceback (most recent call last)
_RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/home/ch250798/miniconda3/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 463, in _process_worker
    r = call_item()
        ^^^^^^^^^^^
  File "/home/ch250798/miniconda3/lib/python3.11/site-packages/joblib/externals/loky/process_executor.py", line 291, in __call__
    return self.fn(*self.args, **self.kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ch250798/miniconda3/lib/python3.11/site-packages/joblib/parallel.py", line 589, in __call__
    return [func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ch250798/miniconda3/lib/python3.11/site-packages/joblib/parallel.py", line 589, in <listcomp>
    return [func(*args, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^
pyo3_runtime.PanicException: Could not open file pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz
"""

The above exception was the direct cause of the following exception:

PanicException                            Traceback (most recent call last)
Cell In[11], line 2
      1 from pycisTopic.pseudobulk_peak_calling import export_pseudobulk
----> 2 bw_paths, bed_paths = export_pseudobulk(input_data = cell_data,
      3                  variable = 'celltype',                                                                     # variable by which to generate pseubulk profiles, in this case we want pseudobulks per celltype
      4                  sample_id_col = 'sample_id',
      5                  chromsizes = chromsizes,
      6                  bed_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bed_files/'),  # specify where pseudobulk_bed_files should be stored
      7                  bigwig_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bw_files/'),# specify where pseudobulk_bw_files should be stored
      8                  path_to_fragments = fragments_dict,                                                        # location of fragment fiels
      9                  n_cpu = 8,                                                                                 # specify the number of cores to use, we use ray for multi processing
     10                  normalize_bigwig = True,
     11                  # remove_duplicates = True,
     12                  temp_dir = os.path.join(tmp_dir, 'ray_spill'),
     13                  split_pattern = '-')

File /temp_work/ch250798/Scenic/pycisTopic/python/pycisTopic/pseudobulk_peak_calling.py:156, in export_pseudobulk(input_data, variable, chromsizes, bed_path, bigwig_path, path_to_fragments, sample_id_col, n_cpu, normalize_bigwig, split_pattern, temp_dir)
    153 # For each sample, get fragments for each cell type
    155 log.info("Splitting fragments by cell type.")
--> 156 split_fragment_files_by_cell_type(
    157     sample_to_fragment_file = path_to_fragments,
    158     path_to_temp_folder = temp_dir,
    159     path_to_output_folder = bed_path,
    160     sample_to_cell_type_to_cell_barcodes = sample_to_cell_type_to_barcodes,
    161     chromsizes = chromsizes_dict,
    162     n_cpu = n_cpu,
    163     verbose = False,
    164     clear_temp_folder = True
    165 )
    167 bed_paths = {}
    168 for cell_type in cell_data[variable].unique():

File ~/miniconda3/lib/python3.11/site-packages/scatac_fragment_tools/library/split/split_fragments_by_cell_type.py:72, in split_fragment_files_by_cell_type(sample_to_fragment_file, path_to_temp_folder, path_to_output_folder, sample_to_cell_type_to_cell_barcodes, chromsizes, n_cpu, verbose, clear_temp_folder)
     70 if verbose:
     71     print("Splitting fragments ...")
---> 72 joblib.Parallel(n_jobs=n_cpu)(
     73     joblib.delayed(_rust_scatac_fragment_tools.split_fragments_by_cell_barcode)
     74         (
     75             path_to_fragments = sample_to_fragment_file[sample],
     76             path_to_output_folder = os.path.join(path_to_temp_folder, sample),
     77             cell_type_to_cell_barcodes = sample_to_cell_type_to_cell_barcodes[sample],
     78             chromsizes = chromsizes,
     79             verbose = verbose
     80         )
     81         for sample in sample_to_cell_type_to_cell_barcodes
     82 )
     84 # Check wether all files were create successfully
     85 # and create a dictionary mapping cell types to fragment files
     86 cell_type_to_fragment_files: Dict[str, List[str]] = {}

File ~/miniconda3/lib/python3.11/site-packages/joblib/parallel.py:1952, in Parallel.__call__(self, iterable)
   1946 # The first item from the output is blank, but it makes the interpreter
   1947 # progress until it enters the Try/Except block of the generator and
   1948 # reach the first `yield` statement. This starts the aynchronous
   1949 # dispatch of the tasks to the workers.
   1950 next(output)
-> 1952 return output if self.return_generator else list(output)

File ~/miniconda3/lib/python3.11/site-packages/joblib/parallel.py:1595, in Parallel._get_outputs(self, iterator, pre_dispatch)
   1592     yield
   1594     with self._backend.retrieval_context():
-> 1595         yield from self._retrieve()
   1597 except GeneratorExit:
   1598     # The generator has been garbage collected before being fully
   1599     # consumed. This aborts the remaining tasks if possible and warn
   1600     # the user if necessary.
   1601     self._exception = True

File ~/miniconda3/lib/python3.11/site-packages/joblib/parallel.py:1699, in Parallel._retrieve(self)
   1692 while self._wait_retrieval():
   1693 
   1694     # If the callback thread of a worker has signaled that its task
   1695     # triggered an exception, or if the retrieval loop has raised an
   1696     # exception (e.g. `GeneratorExit`), exit the loop and surface the
   1697     # worker traceback.
   1698     if self._aborting:
-> 1699         self._raise_error_fast()
   1700         break
   1702     # If the next job is not ready for retrieval yet, we just wait for
   1703     # async callbacks to progress.

File ~/miniconda3/lib/python3.11/site-packages/joblib/parallel.py:1734, in Parallel._raise_error_fast(self)
   1730 # If this error job exists, immediatly raise the error by
   1731 # calling get_result. This job might not exists if abort has been
   1732 # called directly or if the generator is gc'ed.
   1733 if error_job is not None:
-> 1734     error_job.get_result(self.timeout)

File ~/miniconda3/lib/python3.11/site-packages/joblib/parallel.py:736, in BatchCompletionCallBack.get_result(self, timeout)
    730 backend = self.parallel._backend
    732 if backend.supports_retrieve_callback:
    733     # We assume that the result has already been retrieved by the
    734     # callback thread, and is stored internally. It's just waiting to
    735     # be returned.
--> 736     return self._return_or_raise()
    738 # For other backends, the main thread needs to run the retrieval step.
    739 try:

File ~/miniconda3/lib/python3.11/site-packages/joblib/parallel.py:754, in BatchCompletionCallBack._return_or_raise(self)
    752 try:
    753     if self.status == TASK_ERROR:
--> 754         raise self._result
    755     return self._result
    756 finally:

PanicException: Could not open file pbmc_tutorial/data/pbmc_granulocyte_sorted_3k_atac_fragments.tsv.gz

Best, Laki

Aya-Balbaa commented 9 months ago

OSError: AF_UNIX path length cannot exceed 107 bytes:

when running import pickle bed_paths = pickle.load(open(os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bed_files/bed_paths.pkl'), 'rb')) bw_paths = pickle.load(open(os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bed_files/bw_paths.pkl'), 'rb')) from pycisTopic.pseudobulk_peak_calling import peak_calling macs_path='macs2'

Run peak calling

narrow_peaks_dict = peak_calling(macs_path, bed_paths, os.path.join(work_dir, 'scATAC/consensus_peak_calling/MACS/'), genome_size='hs', n_cpu=8, input_format='BEDPE', shift=73, ext_size=146, keep_dup = 'all', q_value = 0.05, _temp_dir = os.path.join(tmp_dir, 'ray_spill'))

MVemon commented 9 months ago

Hello @SeppeDeWinter ,

After downloading the index file, export_pseudobulk() ran well with no errors occurring.

Thanks for all the help with debugging.

MVemon commented 9 months ago

Hello @Aya-Balbaa,

This seems to be a problem with the temp_dir being too long? It's been recorded as a recurring issue here.

Not sure if this helps, but good luck.

laki-spk commented 9 months ago

Hi @SeppeDeWinter

I re-downloaded everything, and it is running now. Thank you for your help.

Best, Laki

SeppeDeWinter commented 9 months ago

Hi All

great to hear that everything is running now.

I will close this issue. Feel free to open a new one if needed.

All the best and good luck with your analyses.

Seppe

Aya-Balbaa commented 9 months ago

RuntimeError: command 'macs2 callpeak --treatment ...................return with error (code 127): b'/bin/sh: 1: macs2: not found\n'

when I activate the scenicplus env I find macs2 in /my_path/miniconda3/envs/scenicplus2/bin/macs2 so it's there. any suggestions please?

SeppeDeWinter commented 9 months ago

Hi @Aya-Balbaa

The function you are running has a macs_path paramter, what is this set to?

All the best,

Seppe

Aya-Balbaa commented 9 months ago

Hi @SeppeDeWinter , I tried 1-'/my-path/miniconda3/envs/scenicplus2/bin/macs2' 2-'macs2'

but non of them worked

SeppeDeWinter commented 9 months ago

Hi @Aya-Balbaa

If you activate your conda environment in a bash shell and type


which macs2

what is the output?

Have you tried reinstalling macs2?

Seppe

Aya-Balbaa commented 9 months ago

Hi @SeppeDeWinter , which macs2 is : '/my-path/miniconda3/envs/scenicplus2/bin/macs2'

I also reinstalled macs via conda install -c bioconda macs2 Unfortunately still not working.

Thank you, Aya

SeppeDeWinter commented 9 months ago

Very strange.

And when you copy paste the MACS2 command macs2 callpeak --treatment ... (with all its parameters), does that work?

All the best,

Seppe

Aya-Balbaa commented 9 months ago

Hi @SeppeDeWinter yes that command works and saves the output in the outputdir. It's really strange. and it's working now in jupyter, I refreshed the whole session. not sure if that helped Thank you for your help

Stack7 commented 1 month ago

hi @SeppeDeWinter on the regard of a different output from the tutorial raised by @elhaam, I have got as output only this:

2024-10-03 13:12:55,304 cisTopic     INFO     Splitting fragments by cell type.
2024-10-03 13:13:34,848 cisTopic     INFO     generating bigwig files

The snippet of the tutorial is the following: 

from pycisTopic.pseudobulk_peak_calling import export_pseudobulk
bw_paths, bed_paths = export_pseudobulk(
    input_data = cell_data,
    variable = 'celltype',                                                                     # variable by which to generate pseubulk profiles, in this case we want pseudobulks per celltype
    sample_id_col = 'sample_id',
    chromsizes = chromsizes,
    bed_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bed_files/'),  # specify where pseudobulk_bed_files should be stored
    bigwig_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bw_files/'),# specify where pseudobulk_bw_files should be stored
    path_to_fragments = fragments_dict,                                                        # location of fragment fiels
    n_cpu = cpus,                                                                                 # specify the number of cores to use, we use ray for multi processing
    normalize_bigwig = True,
#     remove_duplicates = True, generate error
    temp_dir = os.path.join(tmp_dir, 'ray_spill'),
    split_pattern = '-')

Python version of the Conda environment: Python 3.11 Scenic+ version: scenicplus versions: 1.0a1

Stack7 commented 1 month ago

hey @Aya-Balbaa I solved that issue just like this: 

narrow_peaks_dict = peak_calling(macs_path,
    bed_paths,
    os.path.join(work_dir, 'scATAC/consensus_peak_calling/MACS/'),
    genome_size='hs',
    n_cpu=cpus,
    input_format='BEDPE',
    shift=73,
    ext_size=146,
    keep_dup = 'all',
    q_value = 0.05,
    _temp_dir = None )
realmungmung commented 3 weeks ago

hey @Stack7 I got the same output of: 2024-10-03 13:12:55,304 cisTopic INFO Splitting fragments by cell type. 2024-10-03 13:13:34,848 cisTopic INFO generating bigwig files as I ran: from pycisTopic.pseudobulk_peak_calling import export_pseudobulk bw_paths, bed_paths = export_pseudobulk( input_data = cell_data, variable = 'celltype', # variable by which to generate pseubulk profiles, in this case we want pseudobulks per celltype sample_id_col = 'sample_id', chromsizes = chromsizes, bed_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bed_files/'), # specify where pseudobulk_bed_files should be stored bigwig_path = os.path.join(work_dir, 'scATAC/consensus_peak_calling/pseudobulk_bw_files/'),# specify where pseudobulk_bw_files should be stored path_to_fragments = fragments_dict, # location of fragment fiels n_cpu = cpus, # specify the number of cores to use, we use ray for multi processing normalize_bigwig = True, temp_dir = os.path.join(tmp_dir, 'ray_spill'), split_pattern = '-')

Also I changed the tmp_dir to /tmp because I kept getting error saying "/scratch" is read-only

I was wondering if you are able to run the rest of the tutorial successfully since I got more errors later. Thank you!

Stack7 commented 2 weeks ago

Hi! Thanks for answering me! I indeed I had have some progress, however, I needed to merge the old tutorials with some other new version to go on further. more specifically, I refer to this and this. However as in this github issue: there is a naming issue with the fragments.