loosolab / TOBIAS_snakemake

Snakemake pipeline for running TOBIAS analysis
MIT License
5 stars 2 forks source link

Running example data fails with snakemake -- unkonwn error, log attached:::: NotImplementedError: function is not implemented for this dtype: [how->mean,dtype->object] #4

Closed c2b2pss closed 1 year ago

c2b2pss commented 1 year ago

Hi,

Trying to run the example data using snakemake --- but it fails with multiple errors. Log is attached.

Sadly, I cannot figure out what the error is. Any help is appreciated !!!

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
Traceback (most recent call last):
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/groupby/groupby.py", line 1490, in array_func
    result = self.grouper._cython_operation(
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 959, in _cython_operation
    return cy_op.cython_operation(
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 657, in cython_operation
    return self._cython_op_ndim_compat(
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 497, in _cython_op_ndim_compat
    return self._call_cython_op(
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 541, in _call_cython_op
    func = self._get_cython_function(self.kind, self.how, values.dtype, is_numeric)
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 173, in _get_cython_function
    raise NotImplementedError(
NotImplementedError: function is not implemented for this dtype: [how->mean,dtype->object]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/nanops.py", line 1692, in _ensure_numeric
    x = float(x)
ValueError: could not convert string to float: 'BACH2_MA1101.2'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/nanops.py", line 1696, in _ensure_numeric
    x = complex(x)
ValueError: could not convert string to complex: 'BACH2_MA1101.2'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/bin/TOBIAS", line 8, in <module>
    sys.exit(main())
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/tobias/TOBIAS.py", line 154, in main
    args.func(args)     
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/tobias/tools/plot_changes.py", line 94, in run_plotchanges
    table = bindetect.loc[chosen_TFS,].groupby("cluster").mean() #mean of each column
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/groupby/groupby.py", line 1855, in mean
    result = self._cython_agg_general(
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/groupby/groupby.py", line 1507, in _cython_agg_general
    new_mgr = data.grouped_reduce(array_func)
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/internals/managers.py", line 1503, in grouped_reduce
    applied = sb.apply(func)
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/internals/blocks.py", line 329, in apply
    result = func(self.values, **kwargs)
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/groupby/groupby.py", line 1503, in array_func
    result = self._agg_py_fallback(values, ndim=data.ndim, alt=alt)
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/groupby/groupby.py", line 1457, in _agg_py_fallback
    res_values = self.grouper.agg_series(ser, alt, preserve_dtype=True)
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 994, in agg_series
    result = self._aggregate_series_pure_python(obj, func)
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/groupby/ops.py", line 1015, in _aggregate_series_pure_python
    res = func(group)
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/groupby/groupby.py", line 1857, in <lambda>
    alt=lambda x: Series(x).mean(numeric_only=numeric_only),
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/generic.py", line 11563, in mean
    return NDFrame.mean(self, axis, skipna, numeric_only, **kwargs)
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/generic.py", line 11208, in mean
    return self._stat_function(
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/generic.py", line 11165, in _stat_function
    return self._reduce(
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/series.py", line 4671, in _reduce
    return op(delegate, skipna=skipna, **kwds)
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/nanops.py", line 96, in _f
    return f(*args, **kwargs)
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/nanops.py", line 158, in f
    result = alt(values, axis=axis, skipna=skipna, **kwds)
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/nanops.py", line 421, in new_func
    result = func(values, axis=axis, skipna=skipna, mask=mask, **kwargs)
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/nanops.py", line 727, in nanmean
    the_sum = _ensure_numeric(values.sum(axis, dtype=dtype_sum))
  File "/home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_/lib/python3.10/site-packages/pandas/core/nanops.py", line 1699, in _ensure_numeric
    raise TypeError(f"Could not convert {x} to numeric") from err
TypeError: Could not convert BACH2_MA1101.2 to numeric
[Mon Apr 17 14:10:32 2023]
Error in rule lineplot_changes:
    jobid: 25
    input: test_output/TFBS/bindetect_results.txt
    output: test_output/overview/bindetect_results_subset.txt, test_output/overview/most_changed_TFs.txt, test_output/overview/TF_changes.pdf
    log: test_output/logs/TF_changes.log (check log file(s) for error details)
    conda-env: /home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
    shell:
        filter_important_factors.py -in test_output/TFBS/bindetect_results.txt -filter 5 -o test_output/overview/bindetect_results_subset.txt > test_output/overview/most_changed_TFs.txt;TOBIAS PlotChanges --bindetect test_output/TFBS/bindetect_results.txt --TFS test_output/overview/most_changed_TFs.txt --output test_output/overview/TF_changes.pdf > test_output/logs/TF_changes.log
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)
c2b2pss commented 1 year ago

LOG FILE:

`Building DAG of jobs...
Your conda installation is not configured to use strict channel priorities. This is however crucial for having robust and correct environments (for details, see https://conda-forge.org/docs/user/tipsandtricks.html). Please consider to configure strict priorities by executing 'conda config --set channel_priority strict'.
Creating conda environment environments/uropa.yaml...
Downloading and installing remote packages.
Environment for /home/user/TOBIAS_snakemake/environments/uropa.yaml created (location: .snakemake/conda/bd9e0dc4179dc2ab76c5ce61377855a3_)
Creating conda environment environments/wilson.yaml...
Downloading and installing remote packages.
Environment for /home/user/TOBIAS_snakemake/snakefiles/../environments/wilson.yaml created (location: .snakemake/conda/74454b52952ef4bdec144786d90494ce_)
Creating conda environment environments/coverage.yaml...
Downloading and installing remote packages.
Environment for /home/user/TOBIAS_snakemake/environments/coverage.yaml created (location: .snakemake/conda/c6f3c567c779e361fbab11b6d78596da_)
Creating conda environment environments/tools.yaml...
Downloading and installing remote packages.
Environment for /home/user/TOBIAS_snakemake/environments/tools.yaml created (location: .snakemake/conda/12466f25012dc11f812c6306de3db65b_)
Creating conda environment environments/macs.yaml...
Downloading and installing remote packages.
Environment for /home/user/TOBIAS_snakemake/environments/macs.yaml created (location: .snakemake/conda/89a1a8815bdc322986832757bbc559df_)
Creating conda environment environments/tobias.yaml...
Downloading and installing remote packages.
Environment for /home/user/TOBIAS_snakemake/snakefiles/../environments/tobias.yaml created (location: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_)
Using shell: /bin/bash
Provided cores: 6
Rules claiming more threads will be scaled down.
Job stats:
job                        count    min threads    max threads
-----------------------  -------  -------------  -------------
all                            1              1              1
atacorrect                     2              6              6
bindetect                      1              6              6
conditionbam                   2              6              6
convert_global_overview        1              1              1
copy_flatfiles                 1              1              1
copy_motifs                    1              1              1
coverage_bigwig                2              1              1
footprinting                   2              6              6
format_motifs                  1              1              1
get_condition_names            1              1              1
get_fasta_chroms               1              1              1
join_bound                     2              6              6
join_pdfs                      6              1              1
lineplot_changes               1              1              1
macs                           4              1              1
merge_condition_peaks          1              1              1
process_peaks                  2              1              1
sort_peak_names                1              1              1
uropa                          1              6              6
uropa_config                   1              1              1
wilson_app                     1              1              1
wilson_howto                   1              1              1
write_config                   1              1              1
total                         38              1              6

Select jobs to execute...

[Mon Apr 17 14:05:53 2023]
rule write_config:
    output: test_output/config.yaml
    jobid: 1
    reason: Missing output files: test_output/config.yaml
    priority: 100
    resources: tmpdir=/tmp

[Mon Apr 17 14:05:53 2023]
Job 15: Running macs2 with .bam-file: data/Tcell_day1.bam
Reason: Missing output files: test_output/peak_calling/Tcell/Tcell_day1_raw.bed

Activating conda environment: .snakemake/conda/89a1a8815bdc322986832757bbc559df_

[Mon Apr 17 14:05:53 2023]
Job 10: Running macs2 with .bam-file: data/Bcell.bam
Reason: Missing output files: test_output/peak_calling/Bcell/Bcell_raw.bed

[Mon Apr 17 14:05:53 2023]
rule wilson_app:
    input: github.molgen.mpg.de/loosolab/wilson-apps/archive/master.zip
    output: test_output/wilson/app.R, test_output/wilson/wilson_installed.txt
    log: test_output/logs/wilson_installation.log
    jobid: 29
    reason: Missing output files: test_output/wilson/app.R, test_output/wilson/wilson_installed.txt
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/89a1a8815bdc322986832757bbc559df_
Downloading from remote: github.molgen.mpg.de/loosolab/wilson-apps/archive/master.zip
Finished download.
Activating conda environment: .snakemake/conda/74454b52952ef4bdec144786d90494ce_

[Mon Apr 17 14:05:57 2023]
rule get_condition_names:
    output: test_output/wilson/condition_names.txt
    jobid: 28
    reason: Missing output files: test_output/wilson/condition_names.txt
    resources: tmpdir=/tmp

[Mon Apr 17 14:05:57 2023]
rule copy_motifs:
    input: data/individual_motifs/MA0002.2.meme, data/individual_motifs/MA0003.4.meme, data/individual_motifs/MA0004.1.jaspar, data/individual_motifs/MA0007.3.jaspar, data/individual_motifs/MA0009.2.meme, data/individual_motifs/MA0014.3.meme, data/individual_motifs/MA0017.2.meme, data/individual_motifs/MA0018.4.jaspar, data/individual_motifs/MA0024.3.jaspar, data/individual_motifs/MA0025.2.meme, data/individual_motifs/MA0027.2.jaspar, data/individual_motifs/MA0028.2.jaspar, data/individual_motifs/MA0029.1.meme, data/individual_motifs/MA0030.1.jaspar, data/individual_motifs/MA0031.1.jaspar, data/individual_motifs/MA0032.2.jaspar, data/individual_motifs/MA0033.2.jaspar, data/individual_motifs/MA0035.4.jaspar, data/individual_motifs/MA0036.3.jaspar, data/individual_motifs/MA0037.3.jaspar, data/individual_motifs/MA0038.2.jaspar, data/individual_motifs/MA0039.4.jaspar, data/individual_motifs/MA0040.1.jaspar, data/individual_motifs/MA0041.1.jaspar, data/individual_motifs/MA0042.2.jaspar, data/individual_motifs/MA0043.3.jaspar, data/individual_motifs/MA0046.2.jaspar, data/individual_motifs/MA0047.3.jaspar, data/individual_motifs/MA0048.2.meme, data/individual_motifs/MA0050.2.jaspar, data/individual_motifs/MA0052.4.meme, data/individual_motifs/MA0058.3.meme, data/individual_motifs/MA0060.3.meme, data/individual_motifs/MA0062.3.jaspar, data/individual_motifs/MA0083.3.meme, data/individual_motifs/MA0088.2.meme, data/individual_motifs/MA0093.3.meme, data/individual_motifs/MA0095.2.meme, data/individual_motifs/MA0098.3.jaspar, data/individual_motifs/MA0100.3.meme, data/individual_motifs/MA0103.3.meme, data/individual_motifs/MA0138.2.meme, data/individual_motifs/MA0139.1.jaspar, data/individual_motifs/MA0144.2.meme, data/individual_motifs/MA0152.1.meme, data/individual_motifs/MA0154.4.jaspar, data/individual_motifs/MA0464.2.jaspar, data/individual_motifs/MA0466.2.jaspar, data/individual_motifs/MA0470.2.jaspar, data/individual_motifs/MA0473.3.jaspar, data/individual_motifs/MA0486.2.jaspar, data/individual_motifs/MA0490.2.jaspar, data/individual_motifs/MA0496.3.meme, data/individual_motifs/MA0497.1.meme, data/individual_motifs/MA0502.2.meme, data/individual_motifs/MA0506.1.meme, data/individual_motifs/MA0511.2.meme, data/individual_motifs/MA0521.1.meme, data/individual_motifs/MA0522.3.meme, data/individual_motifs/MA0523.1.meme, data/individual_motifs/MA0526.3.meme, data/individual_motifs/MA0592.3.jaspar, data/individual_motifs/MA0606.1.meme, data/individual_motifs/MA0609.2.jaspar, data/individual_motifs/MA0624.1.meme, data/individual_motifs/MA0645.1.jaspar, data/individual_motifs/MA0660.1.meme, data/individual_motifs/MA0678.1.meme, data/individual_motifs/MA0684.2.meme, data/individual_motifs/MA0690.1.meme, data/individual_motifs/MA0754.1.jaspar, data/individual_motifs/MA0768.1.meme, data/individual_motifs/MA0782.2.meme, data/individual_motifs/MA0830.2.meme, data/individual_motifs/MA0834.1.jaspar, data/individual_motifs/MA1101.2.jaspar, data/individual_motifs/MA1108.2.meme, data/individual_motifs/MA1114.1.meme, data/individual_motifs/MA1117.1.meme, data/individual_motifs/MA1124.1.meme, data/individual_motifs/MA1125.1.meme, data/individual_motifs/MA1419.1.jaspar, data/individual_motifs/MA1634.1.jaspar
    output: test_output/motifs/individual/MA0002.2.meme, test_output/motifs/individual/MA0003.4.meme, test_output/motifs/individual/MA0004.1.jaspar, test_output/motifs/individual/MA0007.3.jaspar, test_output/motifs/individual/MA0009.2.meme, test_output/motifs/individual/MA0014.3.meme, test_output/motifs/individual/MA0017.2.meme, test_output/motifs/individual/MA0018.4.jaspar, test_output/motifs/individual/MA0024.3.jaspar, test_output/motifs/individual/MA0025.2.meme, test_output/motifs/individual/MA0027.2.jaspar, test_output/motifs/individual/MA0028.2.jaspar, test_output/motifs/individual/MA0029.1.meme, test_output/motifs/individual/MA0030.1.jaspar, test_output/motifs/individual/MA0031.1.jaspar, test_output/motifs/individual/MA0032.2.jaspar, test_output/motifs/individual/MA0033.2.jaspar, test_output/motifs/individual/MA0035.4.jaspar, test_output/motifs/individual/MA0036.3.jaspar, test_output/motifs/individual/MA0037.3.jaspar, test_output/motifs/individual/MA0038.2.jaspar, test_output/motifs/individual/MA0039.4.jaspar, test_output/motifs/individual/MA0040.1.jaspar, test_output/motifs/individual/MA0041.1.jaspar, test_output/motifs/individual/MA0042.2.jaspar, test_output/motifs/individual/MA0043.3.jaspar, test_output/motifs/individual/MA0046.2.jaspar, test_output/motifs/individual/MA0047.3.jaspar, test_output/motifs/individual/MA0048.2.meme, test_output/motifs/individual/MA0050.2.jaspar, test_output/motifs/individual/MA0052.4.meme, test_output/motifs/individual/MA0058.3.meme, test_output/motifs/individual/MA0060.3.meme, test_output/motifs/individual/MA0062.3.jaspar, test_output/motifs/individual/MA0083.3.meme, test_output/motifs/individual/MA0088.2.meme, test_output/motifs/individual/MA0093.3.meme, test_output/motifs/individual/MA0095.2.meme, test_output/motifs/individual/MA0098.3.jaspar, test_output/motifs/individual/MA0100.3.meme, test_output/motifs/individual/MA0103.3.meme, test_output/motifs/individual/MA0138.2.meme, test_output/motifs/individual/MA0139.1.jaspar, test_output/motifs/individual/MA0144.2.meme, test_output/motifs/individual/MA0152.1.meme, test_output/motifs/individual/MA0154.4.jaspar, test_output/motifs/individual/MA0464.2.jaspar, test_output/motifs/individual/MA0466.2.jaspar, test_output/motifs/individual/MA0470.2.jaspar, test_output/motifs/individual/MA0473.3.jaspar, test_output/motifs/individual/MA0486.2.jaspar, test_output/motifs/individual/MA0490.2.jaspar, test_output/motifs/individual/MA0496.3.meme, test_output/motifs/individual/MA0497.1.meme, test_output/motifs/individual/MA0502.2.meme, test_output/motifs/individual/MA0506.1.meme, test_output/motifs/individual/MA0511.2.meme, test_output/motifs/individual/MA0521.1.meme, test_output/motifs/individual/MA0522.3.meme, test_output/motifs/individual/MA0523.1.meme, test_output/motifs/individual/MA0526.3.meme, test_output/motifs/individual/MA0592.3.jaspar, test_output/motifs/individual/MA0606.1.meme, test_output/motifs/individual/MA0609.2.jaspar, test_output/motifs/individual/MA0624.1.meme, test_output/motifs/individual/MA0645.1.jaspar, test_output/motifs/individual/MA0660.1.meme, test_output/motifs/individual/MA0678.1.meme, test_output/motifs/individual/MA0684.2.meme, test_output/motifs/individual/MA0690.1.meme, test_output/motifs/individual/MA0754.1.jaspar, test_output/motifs/individual/MA0768.1.meme, test_output/motifs/individual/MA0782.2.meme, test_output/motifs/individual/MA0830.2.meme, test_output/motifs/individual/MA0834.1.jaspar, test_output/motifs/individual/MA1101.2.jaspar, test_output/motifs/individual/MA1108.2.meme, test_output/motifs/individual/MA1114.1.meme, test_output/motifs/individual/MA1117.1.meme, test_output/motifs/individual/MA1124.1.meme, test_output/motifs/individual/MA1125.1.meme, test_output/motifs/individual/MA1419.1.jaspar, test_output/motifs/individual/MA1634.1.jaspar
    jobid: 23
    reason: Missing output files: test_output/motifs/individual/MA0606.1.meme, test_output/motifs/individual/MA0042.2.jaspar, test_output/motifs/individual/MA0040.1.jaspar, test_output/motifs/individual/MA0039.4.jaspar, test_output/motifs/individual/MA1634.1.jaspar, test_output/motifs/individual/MA0782.2.meme, test_output/motifs/individual/MA0004.1.jaspar, test_output/motifs/individual/MA0100.3.meme, test_output/motifs/individual/MA0033.2.jaspar, test_output/motifs/individual/MA0018.4.jaspar, test_output/motifs/individual/MA0690.1.meme, test_output/motifs/individual/MA0025.2.meme, test_output/motifs/individual/MA0506.1.meme, test_output/motifs/individual/MA0043.3.jaspar, test_output/motifs/individual/MA1124.1.meme, test_output/motifs/individual/MA0030.1.jaspar, test_output/motifs/individual/MA0496.3.meme, test_output/motifs/individual/MA1108.2.meme, test_output/motifs/individual/MA0152.1.meme, test_output/motifs/individual/MA0103.3.meme, test_output/motifs/individual/MA0035.4.jaspar, test_output/motifs/individual/MA0095.2.meme, test_output/motifs/individual/MA0521.1.meme, test_output/motifs/individual/MA0024.3.jaspar, test_output/motifs/individual/MA0139.1.jaspar, test_output/motifs/individual/MA0592.3.jaspar, test_output/motifs/individual/MA0470.2.jaspar, test_output/motifs/individual/MA0486.2.jaspar, test_output/motifs/individual/MA1125.1.meme, test_output/motifs/individual/MA0088.2.meme, test_output/motifs/individual/MA0678.1.meme, test_output/motifs/individual/MA0029.1.meme, test_output/motifs/individual/MA0046.2.jaspar, test_output/motifs/individual/MA0007.3.jaspar, test_output/motifs/individual/MA0058.3.meme, test_output/motifs/individual/MA0052.4.meme, test_output/motifs/individual/MA0645.1.jaspar, test_output/motifs/individual/MA1419.1.jaspar, test_output/motifs/individual/MA0009.2.meme, test_output/motifs/individual/MA0523.1.meme, test_output/motifs/individual/MA0062.3.jaspar, test_output/motifs/individual/MA0511.2.meme, test_output/motifs/individual/MA1101.2.jaspar, test_output/motifs/individual/MA0497.1.meme, test_output/motifs/individual/MA0660.1.meme, test_output/motifs/individual/MA0003.4.meme, test_output/motifs/individual/MA0050.2.jaspar, test_output/motifs/individual/MA0609.2.jaspar, test_output/motifs/individual/MA1114.1.meme, test_output/motifs/individual/MA0036.3.jaspar, test_output/motifs/individual/MA0060.3.meme, test_output/motifs/individual/MA0041.1.jaspar, test_output/motifs/individual/MA0834.1.jaspar, test_output/motifs/individual/MA0144.2.meme, test_output/motifs/individual/MA0048.2.meme, test_output/motifs/individual/MA0502.2.meme, test_output/motifs/individual/MA0098.3.jaspar, test_output/motifs/individual/MA0138.2.meme, test_output/motifs/individual/MA0014.3.meme, test_output/motifs/individual/MA0047.3.jaspar, test_output/motifs/individual/MA0032.2.jaspar, test_output/motifs/individual/MA0526.3.meme, test_output/motifs/individual/MA0466.2.jaspar, test_output/motifs/individual/MA0037.3.jaspar, test_output/motifs/individual/MA0768.1.meme, test_output/motifs/individual/MA0522.3.meme, test_output/motifs/individual/MA0038.2.jaspar, test_output/motifs/individual/MA0473.3.jaspar, test_output/motifs/individual/MA0490.2.jaspar, test_output/motifs/individual/MA0464.2.jaspar, test_output/motifs/individual/MA0624.1.meme, test_output/motifs/individual/MA0830.2.meme, test_output/motifs/individual/MA0154.4.jaspar, test_output/motifs/individual/MA0002.2.meme, test_output/motifs/individual/MA0754.1.jaspar, test_output/motifs/individual/MA1117.1.meme, test_output/motifs/individual/MA0031.1.jaspar, test_output/motifs/individual/MA0083.3.meme, test_output/motifs/individual/MA0028.2.jaspar, test_output/motifs/individual/MA0027.2.jaspar, test_output/motifs/individual/MA0684.2.meme, test_output/motifs/individual/MA0017.2.meme, test_output/motifs/individual/MA0093.3.meme
    resources: tmpdir=/tmp

[Mon Apr 17 14:05:57 2023]
Finished job 1.
1 of 38 steps (3%) done
[Mon Apr 17 14:05:57 2023]
Finished job 15.
2 of 38 steps (5%) done
[Mon Apr 17 14:05:57 2023]
Finished job 28.
3 of 38 steps (8%) done
Select jobs to execute...

[Mon Apr 17 14:05:57 2023]
Job 16: Running macs2 with .bam-file: data/Tcell_day3.bam
Reason: Missing output files: test_output/peak_calling/Tcell/Tcell_day3_raw.bed

Activating conda environment: .snakemake/conda/89a1a8815bdc322986832757bbc559df_

[Mon Apr 17 14:05:57 2023]
rule copy_flatfiles:
    input: data/genome.fa.gz, data/blacklist.bed, data/genes.gtf
    output: test_output/flatfiles/genome.fa.gz, test_output/flatfiles/blacklist.bed, test_output/flatfiles/genes.gtf
    jobid: 11
    reason: Missing output files: test_output/flatfiles/genome.fa.gz, test_output/flatfiles/genes.gtf, test_output/flatfiles/blacklist.bed
    resources: tmpdir=/tmp

[Mon Apr 17 14:05:57 2023]
Job 14: Running macs2 with .bam-file: data/Tcell_day2.bam
Reason: Missing output files: test_output/peak_calling/Tcell/Tcell_day2_raw.bed

Activating conda environment: .snakemake/conda/89a1a8815bdc322986832757bbc559df_
Select jobs to execute...
[Mon Apr 17 14:05:57 2023]
Finished job 11.
4 of 38 steps (11%) done

[Mon Apr 17 14:05:57 2023]
rule get_fasta_chroms:
    input: test_output/flatfiles/genome.fa.gz
    output: test_output/flatfiles/chromsizes.txt, test_output/flatfiles/chromsizes.bed
    jobid: 12
    reason: Missing output files: test_output/flatfiles/chromsizes.txt, test_output/flatfiles/chromsizes.bed; Input files updated by another job: test_output/flatfiles/genome.fa.gz
    resources: tmpdir=/tmp

[Mon Apr 17 14:05:58 2023]
Finished job 12.
5 of 38 steps (13%) done
Select jobs to execute...
[Mon Apr 17 14:05:59 2023]
Finished job 10.
6 of 38 steps (16%) done

[Mon Apr 17 14:05:59 2023]
Job 9: Processing peaks from condition Bcell
Reason: Missing output files: test_output/peak_calling/Bcell_union.bed; Input files updated by another job: test_output/peak_calling/Bcell/Bcell_raw.bed, test_output/flatfiles/blacklist.bed, test_output/flatfiles/chromsizes.bed

Activating conda environment: .snakemake/conda/12466f25012dc11f812c6306de3db65b_
[Mon Apr 17 14:05:59 2023]
Finished job 23.
7 of 38 steps (18%) done
Select jobs to execute...

[Mon Apr 17 14:05:59 2023]
rule format_motifs:
    input: test_output/motifs/individual/MA0002.2.meme, test_output/motifs/individual/MA0003.4.meme, test_output/motifs/individual/MA0004.1.jaspar, test_output/motifs/individual/MA0007.3.jaspar, test_output/motifs/individual/MA0009.2.meme, test_output/motifs/individual/MA0014.3.meme, test_output/motifs/individual/MA0017.2.meme, test_output/motifs/individual/MA0018.4.jaspar, test_output/motifs/individual/MA0024.3.jaspar, test_output/motifs/individual/MA0025.2.meme, test_output/motifs/individual/MA0027.2.jaspar, test_output/motifs/individual/MA0028.2.jaspar, test_output/motifs/individual/MA0029.1.meme, test_output/motifs/individual/MA0030.1.jaspar, test_output/motifs/individual/MA0031.1.jaspar, test_output/motifs/individual/MA0032.2.jaspar, test_output/motifs/individual/MA0033.2.jaspar, test_output/motifs/individual/MA0035.4.jaspar, test_output/motifs/individual/MA0036.3.jaspar, test_output/motifs/individual/MA0037.3.jaspar, test_output/motifs/individual/MA0038.2.jaspar, test_output/motifs/individual/MA0039.4.jaspar, test_output/motifs/individual/MA0040.1.jaspar, test_output/motifs/individual/MA0041.1.jaspar, test_output/motifs/individual/MA0042.2.jaspar, test_output/motifs/individual/MA0043.3.jaspar, test_output/motifs/individual/MA0046.2.jaspar, test_output/motifs/individual/MA0047.3.jaspar, test_output/motifs/individual/MA0048.2.meme, test_output/motifs/individual/MA0050.2.jaspar, test_output/motifs/individual/MA0052.4.meme, test_output/motifs/individual/MA0058.3.meme, test_output/motifs/individual/MA0060.3.meme, test_output/motifs/individual/MA0062.3.jaspar, test_output/motifs/individual/MA0083.3.meme, test_output/motifs/individual/MA0088.2.meme, test_output/motifs/individual/MA0093.3.meme, test_output/motifs/individual/MA0095.2.meme, test_output/motifs/individual/MA0098.3.jaspar, test_output/motifs/individual/MA0100.3.meme, test_output/motifs/individual/MA0103.3.meme, test_output/motifs/individual/MA0138.2.meme, test_output/motifs/individual/MA0139.1.jaspar, test_output/motifs/individual/MA0144.2.meme, test_output/motifs/individual/MA0152.1.meme, test_output/motifs/individual/MA0154.4.jaspar, test_output/motifs/individual/MA0464.2.jaspar, test_output/motifs/individual/MA0466.2.jaspar, test_output/motifs/individual/MA0470.2.jaspar, test_output/motifs/individual/MA0473.3.jaspar, test_output/motifs/individual/MA0486.2.jaspar, test_output/motifs/individual/MA0490.2.jaspar, test_output/motifs/individual/MA0496.3.meme, test_output/motifs/individual/MA0497.1.meme, test_output/motifs/individual/MA0502.2.meme, test_output/motifs/individual/MA0506.1.meme, test_output/motifs/individual/MA0511.2.meme, test_output/motifs/individual/MA0521.1.meme, test_output/motifs/individual/MA0522.3.meme, test_output/motifs/individual/MA0523.1.meme, test_output/motifs/individual/MA0526.3.meme, test_output/motifs/individual/MA0592.3.jaspar, test_output/motifs/individual/MA0606.1.meme, test_output/motifs/individual/MA0609.2.jaspar, test_output/motifs/individual/MA0624.1.meme, test_output/motifs/individual/MA0645.1.jaspar, test_output/motifs/individual/MA0660.1.meme, test_output/motifs/individual/MA0678.1.meme, test_output/motifs/individual/MA0684.2.meme, test_output/motifs/individual/MA0690.1.meme, test_output/motifs/individual/MA0754.1.jaspar, test_output/motifs/individual/MA0768.1.meme, test_output/motifs/individual/MA0782.2.meme, test_output/motifs/individual/MA0830.2.meme, test_output/motifs/individual/MA0834.1.jaspar, test_output/motifs/individual/MA1101.2.jaspar, test_output/motifs/individual/MA1108.2.meme, test_output/motifs/individual/MA1114.1.meme, test_output/motifs/individual/MA1117.1.meme, test_output/motifs/individual/MA1124.1.meme, test_output/motifs/individual/MA1125.1.meme, test_output/motifs/individual/MA1419.1.jaspar, test_output/motifs/individual/MA1634.1.jaspar
    output: test_output/motifs/all_motifs.txt
    log: test_output/logs/format_motifs.log
    jobid: 22
    reason: Missing output files: test_output/motifs/all_motifs.txt; Input files updated by another job: test_output/motifs/individual/MA0606.1.meme, test_output/motifs/individual/MA0042.2.jaspar, test_output/motifs/individual/MA0039.4.jaspar, test_output/motifs/individual/MA1634.1.jaspar, test_output/motifs/individual/MA0782.2.meme, test_output/motifs/individual/MA0004.1.jaspar, test_output/motifs/individual/MA0100.3.meme, test_output/motifs/individual/MA0033.2.jaspar, test_output/motifs/individual/MA0018.4.jaspar, test_output/motifs/individual/MA0025.2.meme, test_output/motifs/individual/MA0506.1.meme, test_output/motifs/individual/MA0690.1.meme, test_output/motifs/individual/MA0043.3.jaspar, test_output/motifs/individual/MA1124.1.meme, test_output/motifs/individual/MA0030.1.jaspar, test_output/motifs/individual/MA0496.3.meme, test_output/motifs/individual/MA1108.2.meme, test_output/motifs/individual/MA0152.1.meme, test_output/motifs/individual/MA0103.3.meme, test_output/motifs/individual/MA0035.4.jaspar, test_output/motifs/individual/MA0095.2.meme, test_output/motifs/individual/MA0521.1.meme, test_output/motifs/individual/MA0024.3.jaspar, test_output/motifs/individual/MA0139.1.jaspar, test_output/motifs/individual/MA0592.3.jaspar, test_output/motifs/individual/MA0470.2.jaspar, test_output/motifs/individual/MA0486.2.jaspar, test_output/motifs/individual/MA1125.1.meme, test_output/motifs/individual/MA0088.2.meme, test_output/motifs/individual/MA0678.1.meme, test_output/motifs/individual/MA0029.1.meme, test_output/motifs/individual/MA0046.2.jaspar, test_output/motifs/individual/MA0007.3.jaspar, test_output/motifs/individual/MA0058.3.meme, test_output/motifs/individual/MA0052.4.meme, test_output/motifs/individual/MA0645.1.jaspar, test_output/motifs/individual/MA1419.1.jaspar, test_output/motifs/individual/MA0009.2.meme, test_output/motifs/individual/MA0523.1.meme, test_output/motifs/individual/MA0062.3.jaspar, test_output/motifs/individual/MA0511.2.meme, test_output/motifs/individual/MA1101.2.jaspar, test_output/motifs/individual/MA0497.1.meme, test_output/motifs/individual/MA0660.1.meme, test_output/motifs/individual/MA0003.4.meme, test_output/motifs/individual/MA0050.2.jaspar, test_output/motifs/individual/MA0609.2.jaspar, test_output/motifs/individual/MA1114.1.meme, test_output/motifs/individual/MA0036.3.jaspar, test_output/motifs/individual/MA0060.3.meme, test_output/motifs/individual/MA0041.1.jaspar, test_output/motifs/individual/MA0834.1.jaspar, test_output/motifs/individual/MA0017.2.meme, test_output/motifs/individual/MA0144.2.meme, test_output/motifs/individual/MA0048.2.meme, test_output/motifs/individual/MA0502.2.meme, test_output/motifs/individual/MA0098.3.jaspar, test_output/motifs/individual/MA0138.2.meme, test_output/motifs/individual/MA0014.3.meme, test_output/motifs/individual/MA0047.3.jaspar, test_output/motifs/individual/MA0032.2.jaspar, test_output/motifs/individual/MA0526.3.meme, test_output/motifs/individual/MA0466.2.jaspar, test_output/motifs/individual/MA0037.3.jaspar, test_output/motifs/individual/MA0768.1.meme, test_output/motifs/individual/MA0038.2.jaspar, test_output/motifs/individual/MA0522.3.meme, test_output/motifs/individual/MA0473.3.jaspar, test_output/motifs/individual/MA0490.2.jaspar, test_output/motifs/individual/MA0464.2.jaspar, test_output/motifs/individual/MA0624.1.meme, test_output/motifs/individual/MA0830.2.meme, test_output/motifs/individual/MA0154.4.jaspar, test_output/motifs/individual/MA0002.2.meme, test_output/motifs/individual/MA0754.1.jaspar, test_output/motifs/individual/MA1117.1.meme, test_output/motifs/individual/MA0031.1.jaspar, test_output/motifs/individual/MA0083.3.meme, test_output/motifs/individual/MA0028.2.jaspar, test_output/motifs/individual/MA0027.2.jaspar, test_output/motifs/individual/MA0684.2.meme, test_output/motifs/individual/MA0040.1.jaspar, test_output/motifs/individual/MA0093.3.meme
    priority: 2
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:05:59 2023]
Finished job 9.
8 of 38 steps (21%) done
Select jobs to execute...
[Mon Apr 17 14:06:00 2023]
Finished job 14.
9 of 38 steps (24%) done
[Mon Apr 17 14:06:02 2023]
Finished job 22.
10 of 38 steps (26%) done
[Mon Apr 17 14:06:03 2023]
Finished job 16.
11 of 38 steps (29%) done

[Mon Apr 17 14:06:03 2023]
Job 13: Processing peaks from condition Tcell
Reason: Missing output files: test_output/peak_calling/Tcell_union.bed; Input files updated by another job: test_output/peak_calling/Tcell/Tcell_day3_raw.bed, test_output/flatfiles/blacklist.bed, test_output/peak_calling/Tcell/Tcell_day2_raw.bed, test_output/flatfiles/chromsizes.bed, test_output/peak_calling/Tcell/Tcell_day1_raw.bed

Activating conda environment: .snakemake/conda/12466f25012dc11f812c6306de3db65b_
[Mon Apr 17 14:06:04 2023]
Finished job 13.
12 of 38 steps (32%) done
Select jobs to execute...

[Mon Apr 17 14:06:04 2023]
Job 8: Merging peaks across conditions
Reason: Missing output files: test_output/peak_calling/all_merged.tmp; Input files updated by another job: test_output/peak_calling/Bcell_union.bed, test_output/peak_calling/Tcell_union.bed

Activating conda environment: .snakemake/conda/12466f25012dc11f812c6306de3db65b_
[Mon Apr 17 14:06:04 2023]
Finished job 8.
13 of 38 steps (34%) done
Select jobs to execute...

[Mon Apr 17 14:06:04 2023]
rule sort_peak_names:
    input: test_output/peak_calling/all_merged.tmp
    output: test_output/peak_calling/all_merged.bed
    jobid: 7
    reason: Missing output files: test_output/peak_calling/all_merged.bed; Input files updated by another job: test_output/peak_calling/all_merged.tmp
    resources: tmpdir=/tmp

[Mon Apr 17 14:06:06 2023]
Finished job 7.
14 of 38 steps (37%) done
Removing temporary output test_output/peak_calling/all_merged.tmp.
Select jobs to execute...

[Mon Apr 17 14:06:06 2023]
rule uropa_config:
    input: test_output/peak_calling/all_merged.bed, test_output/flatfiles/genes.gtf
    output: test_output/peak_annotation/all_merged_annotated.config
    jobid: 6
    reason: Missing output files: test_output/peak_annotation/all_merged_annotated.config; Input files updated by another job: test_output/peak_calling/all_merged.bed, test_output/flatfiles/genes.gtf
    resources: tmpdir=/tmp

[Mon Apr 17 14:06:09 2023]
Finished job 6.
15 of 38 steps (39%) done
Select jobs to execute...
[Mon Apr 17 14:06:15 2023]
Finished job 29.
16 of 38 steps (42%) done

[Mon Apr 17 14:06:15 2023]
Job 4: Joining individual bamfiles from condition Bcell
Reason: Missing output files: test_output/mapping/Bcell.bam

[Mon Apr 17 14:06:24 2023]
Finished job 4.
17 of 38 steps (45%) done
Select jobs to execute...

[Mon Apr 17 14:06:24 2023]
Job 19: Joining individual bamfiles from condition Tcell
Reason: Missing output files: test_output/mapping/Tcell.bam

[Mon Apr 17 14:06:29 2023]
Finished job 19.
18 of 38 steps (47%) done
Select jobs to execute...

[Mon Apr 17 14:06:29 2023]
rule uropa:
    input: test_output/peak_annotation/all_merged_annotated.config
    output: test_output/peak_annotation/all_merged_annotated_finalhits.txt, test_output/peak_annotation/all_merged_annotated_finalhits_sub.txt, test_output/peak_annotation/all_merged_annotated.bed, test_output/peak_annotation/all_merged_annotated_header.txt
    log: test_output/logs/uropa.log
    jobid: 5
    reason: Missing output files: test_output/peak_annotation/all_merged_annotated.bed, test_output/peak_annotation/all_merged_annotated_header.txt; Input files updated by another job: test_output/peak_annotation/all_merged_annotated.config
    threads: 6
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/bd9e0dc4179dc2ab76c5ce61377855a3_
[Mon Apr 17 14:06:32 2023]
Finished job 5.
19 of 38 steps (50%) done
Select jobs to execute...

[Mon Apr 17 14:06:32 2023]
Job 3: Running ATACorrect for condition Bcell (test_output/mapping/Bcell.bam)
Reason: Missing output files: test_output/bias_correction/Bcell_corrected.bw; Input files updated by another job: test_output/peak_annotation/all_merged_annotated.bed, test_output/mapping/Bcell.bam, test_output/flatfiles/genome.fa.gz

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:07:33 2023]
Finished job 3.
20 of 38 steps (53%) done
Select jobs to execute...

[Mon Apr 17 14:07:33 2023]
Job 2: Running footprinting for condition Bcell (test_output/bias_correction/Bcell_corrected.bw)
Reason: Missing output files: test_output/footprinting/Bcell_footprints.bw; Input files updated by another job: test_output/peak_annotation/all_merged_annotated.bed, test_output/bias_correction/Bcell_corrected.bw

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:08:14 2023]
Finished job 2.
21 of 38 steps (55%) done
Select jobs to execute...

[Mon Apr 17 14:08:14 2023]
Job 18: Running ATACorrect for condition Tcell (test_output/mapping/Tcell.bam)
Reason: Missing output files: test_output/bias_correction/Tcell_corrected.bw; Input files updated by another job: test_output/peak_annotation/all_merged_annotated.bed, test_output/mapping/Tcell.bam, test_output/flatfiles/genome.fa.gz

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:09:16 2023]
Finished job 18.
22 of 38 steps (58%) done
Select jobs to execute...

[Mon Apr 17 14:09:16 2023]
Job 17: Running footprinting for condition Tcell (test_output/bias_correction/Tcell_corrected.bw)
Reason: Missing output files: test_output/footprinting/Tcell_footprints.bw; Input files updated by another job: test_output/peak_annotation/all_merged_annotated.bed, test_output/bias_correction/Tcell_corrected.bw

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:09:57 2023]
Finished job 17.
23 of 38 steps (61%) done
Select jobs to execute...

[Mon Apr 17 14:09:57 2023]
Job 21: Running BINDetect
Reason: Missing output files: test_output/TFBS; Input files updated by another job: test_output/peak_annotation/all_merged_annotated.bed, test_output/motifs/all_motifs.txt, test_output/flatfiles/genome.fa.gz, test_output/footprinting/Tcell_footprints.bw, test_output/footprinting/Bcell_footprints.bw, test_output/peak_annotation/all_merged_annotated_header.txt
DAG of jobs will be updated after completion.

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:10:18 2023]
Finished job 21.
24 of 38 steps (63%) done
Select jobs to execute...

[Mon Apr 17 14:10:21 2023]
Job 435: Plotting comparison of cutsite signals for "OLIG2_MA0678.1" between conditions
Reason: Missing output files: test_output/TFBS/OLIG2_MA0678.1/plots/OLIG2_MA0678.1_aggregate_comparison_all.pdf, test_output/TFBS/OLIG2_MA0678.1/plots/OLIG2_MA0678.1_aggregate_comparison_bound.pdf

[Mon Apr 17 14:10:21 2023]
Job 405: Plotting comparison of cutsite signals for "PKNOX1_MA0782.2" between conditions
Reason: Missing output files: test_output/TFBS/PKNOX1_MA0782.2/plots/PKNOX1_MA0782.2_aggregate_comparison_bound.pdf, test_output/TFBS/PKNOX1_MA0782.2/plots/PKNOX1_MA0782.2_aggregate_comparison_all.pdf

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_

[Mon Apr 17 14:10:21 2023]
Job 408: Plotting comparison of cutsite signals for "RUNX3_MA0684.2" between conditions
Reason: Missing output files: test_output/TFBS/RUNX3_MA0684.2/plots/RUNX3_MA0684.2_aggregate_comparison_all.pdf, test_output/TFBS/RUNX3_MA0684.2/plots/RUNX3_MA0684.2_aggregate_comparison_bound.pdf

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:10:21 2023]
Job 422: Plotting comparison of cutsite signals for "TCF3_MA0522.3" between conditions
Reason: Missing output files: test_output/TFBS/TCF3_MA0522.3/plots/TCF3_MA0522.3_aggregate_comparison_all.pdf, test_output/TFBS/TCF3_MA0522.3/plots/TCF3_MA0522.3_aggregate_comparison_bound.pdf

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_

[Mon Apr 17 14:10:21 2023]
Job 419: Plotting comparison of cutsite signals for "EBF1_MA0154.4" between conditions
Reason: Missing output files: test_output/TFBS/EBF1_MA0154.4/plots/EBF1_MA0154.4_aggregate_comparison_bound.pdf, test_output/TFBS/EBF1_MA0154.4/plots/EBF1_MA0154.4_aggregate_comparison_all.pdf

[Mon Apr 17 14:10:21 2023]
Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
Job 433: Plotting comparison of cutsite signals for "Tcf12_MA0521.1" between conditions
Reason: Missing output files: test_output/TFBS/Tcf12_MA0521.1/plots/Tcf12_MA0521.1_aggregate_comparison_all.pdf, test_output/TFBS/Tcf12_MA0521.1/plots/Tcf12_MA0521.1_aggregate_comparison_bound.pdf

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:10:25 2023]
Finished job 435.
25 of 536 steps (5%) done
Select jobs to execute...

[Mon Apr 17 14:10:25 2023]
Job 456: Plotting comparison of cutsite signals for "ZNF384_MA1125.1" between conditions
Reason: Missing output files: test_output/TFBS/ZNF384_MA1125.1/plots/ZNF384_MA1125.1_aggregate_comparison_bound.pdf, test_output/TFBS/ZNF384_MA1125.1/plots/ZNF384_MA1125.1_aggregate_comparison_all.pdf

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:10:25 2023]
Finished job 419.
26 of 536 steps (5%) done
Select jobs to execute...

[Mon Apr 17 14:10:25 2023]
Job 457: Plotting comparison of cutsite signals for "GFI1_MA0038.2" between conditions
Reason: Missing output files: test_output/TFBS/GFI1_MA0038.2/plots/GFI1_MA0038.2_aggregate_comparison_bound.pdf, test_output/TFBS/GFI1_MA0038.2/plots/GFI1_MA0038.2_aggregate_comparison_all.pdf

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:10:25 2023]
Finished job 408.
27 of 536 steps (5%) done
Select jobs to execute...

[Mon Apr 17 14:10:26 2023]
Job 458: Plotting comparison of cutsite signals for "Foxd3_MA0041.1" between conditions
Reason: Missing output files: test_output/TFBS/Foxd3_MA0041.1/plots/Foxd3_MA0041.1_aggregate_comparison_bound.pdf, test_output/TFBS/Foxd3_MA0041.1/plots/Foxd3_MA0041.1_aggregate_comparison_all.pdf

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:10:26 2023]
Finished job 405.
28 of 536 steps (5%) done
Select jobs to execute...

[Mon Apr 17 14:10:26 2023]
Job 459: Plotting comparison of cutsite signals for "NFIL3_MA0025.2" between conditions
Reason: Missing output files: test_output/TFBS/NFIL3_MA0025.2/plots/NFIL3_MA0025.2_aggregate_comparison_bound.pdf, test_output/TFBS/NFIL3_MA0025.2/plots/NFIL3_MA0025.2_aggregate_comparison_all.pdf

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:10:26 2023]
Finished job 433.
29 of 536 steps (5%) done
Select jobs to execute...

[Mon Apr 17 14:10:26 2023]
Job 460: Plotting comparison of cutsite signals for "ZEB1_MA0103.3" between conditions
Reason: Missing output files: test_output/TFBS/ZEB1_MA0103.3/plots/ZEB1_MA0103.3_aggregate_comparison_all.pdf, test_output/TFBS/ZEB1_MA0103.3/plots/ZEB1_MA0103.3_aggregate_comparison_bound.pdf

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:10:26 2023]
Finished job 422.
30 of 536 steps (6%) done
Select jobs to execute...

[Mon Apr 17 14:10:26 2023]
Job 461: Plotting comparison of cutsite signals for "Mecom_MA0029.1" between conditions
Reason: Missing output files: test_output/TFBS/Mecom_MA0029.1/plots/Mecom_MA0029.1_aggregate_comparison_all.pdf, test_output/TFBS/Mecom_MA0029.1/plots/Mecom_MA0029.1_aggregate_comparison_bound.pdf

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:10:30 2023]
Finished job 458.
31 of 536 steps (6%) done
Select jobs to execute...

[Mon Apr 17 14:10:30 2023]
rule lineplot_changes:
    input: test_output/TFBS/bindetect_results.txt
    output: test_output/overview/bindetect_results_subset.txt, test_output/overview/most_changed_TFs.txt, test_output/overview/TF_changes.pdf
    log: test_output/logs/TF_changes.log
    jobid: 25
    reason: Missing output files: test_output/overview/TF_changes.pdf
    priority: 2
    resources: tmpdir=/tmp

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:10:30 2023]
Finished job 459.
32 of 536 steps (6%) done
Select jobs to execute...

[Mon Apr 17 14:10:30 2023]
Job 472: Plotting comparison of cutsite signals for "TBXT_MA0009.2" between conditions
Reason: Missing output files: test_output/TFBS/TBXT_MA0009.2/plots/TBXT_MA0009.2_aggregate_comparison_bound.pdf, test_output/TFBS/TBXT_MA0009.2/plots/TBXT_MA0009.2_aggregate_comparison_all.pdf

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:10:30 2023]
Finished job 457.
33 of 536 steps (6%) done
[Mon Apr 17 14:10:30 2023]
Finished job 456.
34 of 536 steps (6%) done
Select jobs to execute...

[Mon Apr 17 14:10:30 2023]
Job 453: Plotting comparison of cutsite signals for "MAX_MA0058.3" between conditions
Reason: Missing output files: test_output/TFBS/MAX_MA0058.3/plots/MAX_MA0058.3_aggregate_comparison_all.pdf, test_output/TFBS/MAX_MA0058.3/plots/MAX_MA0058.3_aggregate_comparison_bound.pdf

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_

[Mon Apr 17 14:10:30 2023]
Job 447: Plotting comparison of cutsite signals for "Foxq1_MA0040.1" between conditions
Reason: Missing output files: test_output/TFBS/Foxq1_MA0040.1/plots/Foxq1_MA0040.1_aggregate_comparison_all.pdf, test_output/TFBS/Foxq1_MA0040.1/plots/Foxq1_MA0040.1_aggregate_comparison_bound.pdf

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:10:30 2023]
Finished job 461.
35 of 536 steps (7%) done
Select jobs to execute...

[Mon Apr 17 14:10:30 2023]
Job 468: Plotting comparison of cutsite signals for "USF2_MA0526.3" between conditions
Reason: Missing output files: test_output/TFBS/USF2_MA0526.3/plots/USF2_MA0526.3_aggregate_comparison_bound.pdf, test_output/TFBS/USF2_MA0526.3/plots/USF2_MA0526.3_aggregate_comparison_all.pdf

Select jobs to execute...
Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:10:31 2023]
Finished job 460.
36 of 536 steps (7%) done

[Mon Apr 17 14:10:31 2023]
Job 469: Plotting comparison of cutsite signals for "NHLH1_MA0048.2" between conditions
Reason: Missing output files: test_output/TFBS/NHLH1_MA0048.2/plots/NHLH1_MA0048.2_aggregate_comparison_bound.pdf, test_output/TFBS/NHLH1_MA0048.2/plots/NHLH1_MA0048.2_aggregate_comparison_all.pdf

Activating conda environment: .snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
[Mon Apr 17 14:10:32 2023]
Error in rule lineplot_changes:
    jobid: 25
    input: test_output/TFBS/bindetect_results.txt
    output: test_output/overview/bindetect_results_subset.txt, test_output/overview/most_changed_TFs.txt, test_output/overview/TF_changes.pdf
    log: test_output/logs/TF_changes.log (check log file(s) for error details)
    conda-env: /home/user/TOBIAS_snakemake/.snakemake/conda/b64f8cafc44f6fd604e0d4c62f7a507d_
    shell:
        filter_important_factors.py -in test_output/TFBS/bindetect_results.txt -filter 5 -o test_output/overview/bindetect_results_subset.txt > test_output/overview/most_changed_TFs.txt;TOBIAS PlotChanges --bindetect test_output/TFBS/bindetect_results.txt --TFS test_output/overview/most_changed_TFs.txt --output test_output/overview/TF_changes.pdf > test_output/logs/TF_changes.log
        (one of the commands exited with non-zero exit code; note that snakemake uses bash strict mode!)

Removing output files of failed job lineplot_changes since they might be corrupted:
test_output/overview/bindetect_results_subset.txt, test_output/overview/most_changed_TFs.txt, test_output/overview/TF_changes.pdf
[Mon Apr 17 14:10:35 2023]
Finished job 453.
37 of 536 steps (7%) done
[Mon Apr 17 14:10:35 2023]
Finished job 472.
38 of 536 steps (7%) done
[Mon Apr 17 14:10:35 2023]
Finished job 468.
39 of 536 steps (7%) done
[Mon Apr 17 14:10:35 2023]
Finished job 447.
40 of 536 steps (7%) done
[Mon Apr 17 14:10:35 2023]
Finished job 469.
41 of 536 steps (8%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message
Complete log: .snakemake/log/2023-04-17T135850.654639.snakemake.log~
aksarkar commented 1 year ago

I had the same problem. However, I found that running the job manually worked

conda activate .snakemake/conda/cb62d560d34b3f8ac6ac84509d05b4bb_
TOBIAS PlotChanges --bindetect test_output/TFBS/bindetect_results.txt --TFS test_output/overview/most_changed_TFs.txt --output test_output/overview/TF_changes.pdf > test_output/logs/TF_changes.log
conda deactivate

after which the remainder of the pipeline completed successfully.