Closed steffenheyne closed 8 years ago
plotHeatmap ... --outFileSortedRegions test_sorted.bed
$ head test_sorted.bed
11 54971969 54998579 gene_r160 0.0 - 54971969 54971969 0 1 0 26610 DE_oldKO_down 6 149357407 149507379 gene_r800 0.0 + 149357407 149357407 0 1 0 149972 DE_oldKO_down 6 149357407 149507379 gene_r799 0.0 + 149357407 149357407 0 1 0 149972 DE_oldKO_down 3 159502658 159598139 gene_r329 0.0 + 159502658 159502658 0 1 0 95481 DE_oldKO_down
col5 is always 0.0 and file is not sorted
computeMatrix ... --outFileSortedRegions test_sorted.bed
Traceback (most recent call last):
File "/home/heyne/install/miniconda3/bin/computeMatrix", line 7, in
plotheatmap ... --outFileNameMatrix test_matrix.tab
Traceback (most recent call last): File "/home/heyne/install/miniconda3/lib/python3.5/site-packages/numpy/lib/npyio.py", line 1158, in savetxt fh.write(asbytes(format % tuple(row) + newline)) TypeError: write() argument must be str, not bytes
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/heyne/install/miniconda3/bin/plotHeatmap", line 7, in
I'll add that the columns 11 and 12 from --outFileSortedRegions
are swapped.
Regarding column 5 in the output of plotHeatmap ... --outFileSortedRegions test_sorted.bed
, it will always be 0.0 if either there were no values in the input file or they were all 0.0 to begin with. The value there has absolutely no relation to any computation done by deepTools.
I can't reproduce the computeMatrix ... --outFileNameMatrix test_matrix.tab
issue.
I have a branch with computeMatrix ... --outFileSortedRegions test_sorted.bed
fixed. It also has the issue that I noted fixed (though I expect this to break something in plotHeatmap/plotProfile, though that should be simple to fix).
I've now also fixed the plotheatmap ... --outFileNameMatrix test_matrix.tab
issue. You can test these in the feature/fix_364
branch.
If you can get me a reproducible example of the computeMatrix ... --outFileNameMatrix
issue with that then I'll fix it. Otherwise I'll close this before 2.3.0 comes out.
ok, I will test and also look for a reproducer... But could you reproduce that "--outFileSortedRegions" creates a non-sorted bed file, both computeMatrix and plotHeatmap? (...while using "--sortRegions")
For plotHeatmap they appear to be sorted. For computeMatrix they appear to not be (I'm not sure why, I still need to fix that).
The default for --sortRegions
differs between computeMatrix
(default: no
) and plotHeatmap
: (default: ascend
). If you specify what you want in computeMatrix then it'll do it. I don't know the rationale for the difference in defaults.
These defaults might be ok, just avoid sorting the "raw matrix", but creating a nice and sorted heatmap is ok to some point. But I remember that I got even with "--sortRegions descend" a not properly sorted bed file (don't no if it was computeMatrix or plotHeatmap), at least when I used the bed file in plotHeatmap again and this time with "--sortRegions no" and the heatmap looked not properly sorted...
Fix #364 does not fix:
plotHeatmap -m test_sorted.matrix -out test_pHM.sorted.png --outFileSortedRegions test_sorted.pHM.bed --sortRegions ascend
computeMatrix reference-point -R test_sorted.pHM.bed -S /data/manke/repository/deepTools_testDataset/H3K27Me3.bigWig /data/manke/repository/deepTools_testDataset/H3K4Me1.bigWig -out test_sorted.matrix -b 3000 -a 3000 -bs 100 -p 20 --sortRegions no
plotHeatmap -m test_sorted.matrix -out test_pHM.sorted.png --sortRegions no
sorted bed from plotHeatmap -> computeMatrix with sorted bed, no sorting on matrix -> plotHeatmap with matrix gives "unsorted" heatmap rather then ascending sorted heatmap
and reproducer for the computeMatrix issue (without local paths but on branch #364 ) :
computeMatrix reference-point -R genes19.bed -S H3K27Me3.bigWig H3K4Me1.bigWig -out test.matrix -b 3000 -a 3000 -bs 100 -p 20 --outFileSortedRegions test.bed
Traceback (most recent call last):
File "/home/heyne/install/deepTools/bin/computeMatrix", line 7, in <module>
main()
File "/package/deeptools-develop/lib/python3.4/site-packages/deeptools/computeMatrix.py", line 399, in main
hm.save_BED(args.outFileSortedRegions)
File "/package/deeptools-develop/lib/python3.4/site-packages/deeptools/heatmapper.py", line 914, in save_BED
starts = region['start'].split(",")
TypeError: list indices must be integers, not str
computeMatrix with --outFileNameMatrix also not works on #364
computeMatrix reference-point -R genes19.bed -S H3K27Me3.bigWig H3K4Me1.bigWig -out test.matrix -b 3000 -a 3000 -bs 100 -p 20 --outFileNameMatrix test.mat
Traceback (most recent call last):
File "/package/deeptools-develop/lib/python3.4/site-packages/numpy/lib/npyio.py", line 1158, in savetxt
fh.write(asbytes(format % tuple(row) + newline))
TypeError: must be str, not bytes
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/heyne/install/deepTools/bin/computeMatrix", line 7, in <module>
main()
File "/package/deeptools-develop/lib/python3.4/site-packages/deeptools/computeMatrix.py", line 396, in main
hm.save_matrix_values(args.outFileNameMatrix)
File "/package/deeptools-develop/lib/python3.4/site-packages/deeptools/heatmapper.py", line 899, in save_matrix_values
np.savetxt(fh, self.matrix.matrix, fmt="%.4g")
File "/package/deeptools-develop/lib/python3.4/site-packages/numpy/lib/npyio.py", line 1162, in savetxt
% (str(X.dtype), format))
TypeError: Mismatch between array dtype ('float64') and format specifier ('%.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g')
ah, no wonder about the paths "/package/deeptools-develop/...", if I "module unload deeptools/develop" and then run, it prints same error, this time with paths to my installed #364 version...
I was slightly annoyed after the game, so I fixed the --outFileNameMatrix and --outFileSortedRegions issues in python3 and tested the fixed versions in python2.
Regarding the custom sorted BED input to computeMatrix, that's no longer supported. You can sort a BED file (or GTF file, or both) however you want, the only thing that's honored is group boundaries. That that worked previously was due to computeMatrix having its own parallelization code. It made much more sense to unify behind the same code for everything in deepTools, so it's easier to fix bugs and add features. If you need the output sorted in a particular way for some reason then it'd be simpler to make a separate script for that, as I've done in the riboseq branch.
computeMatrix ... --outFileNameMatrix test_matrix.tab gives this
Traceback (most recent call last): File "/home/heyne/install/miniconda3/lib/python3.5/site-packages/numpy/lib/npyio.py", line 1158, in savetxt fh.write(asbytes(format % tuple(row) + newline)) TypeError: write() argument must be str, not bytes
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "/home/heyne/install/miniconda3/bin/computeMatrix", line 7, in
main()
File "/home/heyne/install/miniconda3/lib/python3.5/site-packages/deeptools/computeMatrix.py", line 396, in main
hm.save_matrix_values(args.outFileNameMatrix)
File "/home/heyne/install/miniconda3/lib/python3.5/site-packages/deeptools/heatmapper.py", line 901, in save_matrix_values
np.savetxt(fh, self.matrix.matrix, fmt="%.4g")
File "/home/heyne/install/miniconda3/lib/python3.5/site-packages/numpy/lib/npyio.py", line 1162, in savetxt
% (str(X.dtype), format))
TypeError: Mismatch between array dtype ('float64') and format specifier ('%.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g %.4g')