Open sckinta opened 6 years ago
Hi, could you pass in the tail end (or full) BDS log file? (Should end in *.log). Looks like a unicode error when the ATAQC module is writing out the html output, but would help to know exactly which function in the module has the problem. It's likely due to the server update, but good to fix this for anyone else who might run into this on other similar server environments.
Hi. Thank you for quick reply.
Here are the last 20 lines of bds.log.
$ tail -n 20 bds.log
File "/mnt/isilon/sfgi/programs/atac_dnase_pipelines/ataqc/run_ataqc.py", line 1598, in <module>
main()
File "/mnt/isilon/sfgi/programs/atac_dnase_pipelines/ataqc/run_ataqc.py", line 1454, in main
raw_peak_summ, raw_peak_dist = get_region_size_metrics(PEAKS)
File "/mnt/isilon/sfgi/programs/atac_dnase_pipelines/ataqc/run_ataqc.py", line 752, in get_region_size_metrics
ax.set_title('Peak width distribution for {0}'.format(filename))
File "/mnt/isilon/sfgi/programs/miniconda3/envs/bds_atac/lib/python2.7/site-packages/matplotlib/axes/_axes.py", line 172, in set_title
title.set_text(label)
File "/mnt/isilon/sfgi/programs/miniconda3/envs/bds_atac/lib/python2.7/site-packages/matplotlib/text.py", line 1206, in set_text
self._text = '%s' % (s,)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 30: ordinal not in range(128)
Fatal error: /mnt/isilon/sfgi/programs/atac_dnase_pipelines/atac.bds, line 1612, pos 2. Task/s failed.
atac.bds, line 82 : main()
atac.bds, line 85 : void main() { // atac pipeline starts here
atac.bds, line 109 : ataqc()
atac.bds, line 1601 : void ataqc() {
atac.bds, line 1612 : wait
Creating checkpoint file: Config or command line option disabled checkpoint file creation, nothing done.
Thanks for the log - that was helpful! It looks like there's some issue with setting up the filename, which I'm extrapolating to think that there may be some nonstandard character in your prefix set up (causing the ASCII/Unicode issue). This is something that our code should handle (rather than the user), and I will input a fix for it, but it will take a few days for me to get to it properly - in the meantime, you are welcome to try changing your input prefix and seeing if that resolves your problem, otherwise stay tuned on this issue for the fix to come through. thanks for bringing our attention to it!
Thank you! I think you are right about nonstandard character, since the name for the library is Naïve instead of naive....
But now, I have a new issue, which happened to all the libraries I am working on. The error is ValueError: all the input array dimensions except for the concatenation axis must match exactly
It looks like "_nx.concatenate()" in some python files caused problems. I could not figure out why I did not see this error in my successfully run before. Do you have any way to fix this problem too.
Here are more detailed standard error from run_atacseq.py
--------------------Stderr--------------------
Picked up _JAVA_OPTIONS: -Xms256M -Xmx45G -XX:ParallelGCThreads=1 -Djava.io.tmpdir=/mnt/isilon/sfgi/suc1/tmp
ERROR 2018-04-24 10:24:41 ProcessExecutor Warning messages:
ERROR 2018-04-24 10:24:41 ProcessExecutor 1: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH, :
ERROR 2018-04-24 10:24:41 ProcessExecutor zero-length arrow is of indeterminate angle and so skipped
ERROR 2018-04-24 10:24:41 ProcessExecutor 2: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH, :
ERROR 2018-04-24 10:24:41 ProcessExecutor zero-length arrow is of indeterminate angle and so skipped
ERROR 2018-04-24 10:24:41 ProcessExecutor 3: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH, :
ERROR 2018-04-24 10:24:41 ProcessExecutor zero-length arrow is of indeterminate angle and so skipped
ERROR 2018-04-24 10:24:41 ProcessExecutor 4: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH, :
ERROR 2018-04-24 10:24:41 ProcessExecutor zero-length arrow is of indeterminate angle and so skipped
ERROR 2018-04-24 10:24:41 ProcessExecutor 5: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH, :
ERROR 2018-04-24 10:24:41 ProcessExecutor zero-length arrow is of indeterminate angle and so skipped
ERROR 2018-04-24 10:24:41 ProcessExecutor 6: In arrows(metrics$GC, metrics$NORMALIZED_COVERAGE - metrics$ERROR_BAR_WIDTH, :
ERROR 2018-04-24 10:24:41 ProcessExecutor zero-length arrow is of indeterminate angle and so skipped
Picked up _JAVA_OPTIONS: -Xms256M -Xmx45G -XX:ParallelGCThreads=1 -Djava.io.tmpdir=/mnt/isilon/sfgi/suc1/tmp
[bam_sort_core] merging from 20 files...
[bam_sort_core] merging from 17 files...
Picked up _JAVA_OPTIONS: -Xms256M -Xmx45G -XX:ParallelGCThreads=1 -Djava.io.tmpdir=/mnt/isilon/sfgi/suc1/tmp
Picked up _JAVA_OPTIONS: -Xms256M -Xmx45G -XX:ParallelGCThreads=1 -Djava.io.tmpdir=/mnt/isilon/sfgi/suc1/tmp
processing chromosomes
Traceback (most recent call last):
File "/mnt/isilon/sfgi/programs/atac_dnase_pipelines/ataqc/run_ataqc.py", line 1598, in
main()
File "/mnt/isilon/sfgi/programs/atac_dnase_pipelines/ataqc/run_ataqc.py", line 1460, in main
ROADMAP_META, OUTPUT_PREFIX)
File "/mnt/isilon/sfgi/programs/atac_dnase_pipelines/ataqc/run_ataqc.py", line 905, in compare_to_roadmap
sample_mean0_col)
File "/mnt/isilon/sfgi/programs/miniconda3/envs/bds_atac/lib/python2.7/site-packages/scipy/stats/stats.py", line 3310, in spearmanr
rs = np.corrcoef(ar, br, rowvar=axisout)
File "/mnt/isilon/sfgi/programs/miniconda3/envs/bds_atac/lib/python2.7/site-packages/numpy/lib/function_base.py", line 2145, in corrcoef
c = cov(x, y, rowvar)
File "/mnt/isilon/sfgi/programs/miniconda3/envs/bds_atac/lib/python2.7/site-packages/numpy/lib/function_base.py", line 2024, in cov
X = np.vstack((X, y))
File "/mnt/isilon/sfgi/programs/miniconda3/envs/bds_atac/lib/python2.7/site-packages/numpy/core/shape_base.py", line 230, in vstack
return _nx.concatenate([atleast_2d(_m) for _m in tup], 0)
ValueError: all the input array dimensions except for the concatenation axis must match exactly
interesting - it could be related to a different output format from a different version of ucsc tools - can you provide the top of the *signal file in the qc folder if you have it?
Hi,
I ran the pipeline successfully before, but recently I got an error at atacqc step on all my runs. No qc summary (json/html) file was created in qc directory. I am wondering whether anyone has the same issue.
Below is server info that may help you for debugging.
PS: our cluster server has experienced several updates since my last successful run. I do not know what caused the problem here.
Thank you, Chun