search

salzman-lab / SpliZ

Nextflow implementation of SpliZ
MIT License
12 stars 7 forks source link

Timing out of SUMMARIZE_RESULTS #3

Closed apdavid closed 3 years ago

apdavid commented 3 years ago

I got the following error running a separate dataset. I have been able to run this workflow on two other datasets without issue so far, and now I get this error. It appears like the last step (SUMMARIZE_RESULTS) is timing out, the rest of the processes run to completion.

Error executing process > 'NFCORE_SPLIZ:SPLIZ_PIPELINE:ANALYSIS:SUMMARIZE_RESULTS (kubota-cochlea-comp)'

Caused by:
  Process exceeded running time limit (1h)

Command executed:

  final_summary.py \
      --perm_pvals kubota-cochlea-comp_pvals_free_annotation-compartment_100_S_0.1_z_0.0_b_5_SICILIAN.tsv \
      --first_evec first_evec_kubota-cochlea-comp_pvals_free_annotation-compartment_100_S_0.1_z_0.0_b_5_SICILIAN.tsv \
      --second_evec second_evec_kubota-cochlea-comp_pvals_free_annotation-compartment_100_S_0.1_z_0.0_b_5_SICILIAN.tsv \
      --third_evec third_evec_kubota-cochlea-comp_pvals_free_annotation-compartment_100_S_0.1_z_0.0_b_5_SICILIAN.tsv \
      --splizvd kubota-cochlea-comp_sym_SVD_normdonor_S_0.1_z_0.0_b_5_SICILIAN_subcol.tsv \
      --grouping_level_2 free_annotation \
      --grouping_level_1 compartment \
      --outname summary_kubota-cochlea-comp_free_annotation-compartment_S_0.1_z_0.0_b_5_SICILIAN.tsv \
      --outname_log summarize_results.log

Command exit status:
  -

Command output:
  (empty)

Command error:
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)
  /wynton/home/tjan/adavid/.conda/envs/nf_spliz_env/lib/python3.9/site-packages/numpy/lib/nanfunctions.py:1117: RuntimeWarning: Mean of empty slice
    return np.nanmean(a, axis, out=out, keepdims=keepdims)

Work dir:
  /wynton/home/tjan/adavid/kubota/work/c6/9e6e6087df1e227cb74ea553db3757

Tip: view the complete command output by changing to the process work dir and entering the command `cat .command.out`
kaitlinchaung commented 3 years ago

Hmm odd, this step should be retrying with double the memory+time if it runs out. But to be safe, I increased the memory label of that step to 2hours base. Let me know if that doesn't work!

apdavid commented 3 years ago

Yes. this worked. thanks for fixing my issue