search

deeptools / HiCExplorer

HiCExplorer is a powerful and easy to use set of tools to process, normalize and visualize Hi-C data.
https://hicexplorer.readthedocs.org
GNU General Public License v3.0
231 stars 70 forks source link

hicplotTAD error #764

Closed wushyer closed 2 years ago

wushyer commented 2 years ago

Hi I met one error with the latest hicplotTAD. Do you have any idea about this?

Thanks Shuangyang

(base) [shuangyang.wu@clip-login-0 merged]$ hicPlotTADs --tracks conf --region 1:100000-500000 --dpi 300 --outFileName Col_TADs.png --title "Col TADs on chr1" INFO:numexpr.utils:Note: detected 76 virtual cores but NumExpr set to maximum of 64, check "NUMEXPR_MAX_THREADS" environment variable. INFO:numexpr.utils:Note: NumExpr detected 76 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8. INFO:numexpr.utils:NumExpr defaulting to 8 threads.

title not set for 'section 2. [tads]' INFO:hicmatrix.HiCMatrix:Number of poor regions to remove: 92 {'1': 42, '2': 8, '3': 5, '4': 11, '5': 26} INFO:hicmatrix.HiCMatrix:found existing 92 nan bins that will be included for masking WARNING:hicmatrix.HiCMatrix:Bin size is not homogeneous. Median 40000

INFO:pygenometracks.tracksClass:time initializing track(s): INFO:pygenometracks.tracksClass:2.8355629444122314 DEBUG:pygenometracks.tracksClass:Figure size in cm is 40 x 23.6. Dpi is set to 300

INFO:pygenometracks.tracksClass:plotting 1. [hic] INFO:pygenometracks.tracks.GenomeTrack:setting min, max values for track 1. [hic] to: 84.37145272509095, 1146.9949544480594

Traceback (most recent call last): File "/users/shuangyang.wu/miniconda3/bin/hicPlotTADs", line 7, in main() File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/hicexplorer/hicPlotTADs.py", line 9, in main plotTracks.main(args) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/pygenometracks/plotTracks.py", line 307, in main trp.plot(args.outFileName, region, title=args.title) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/pygenometracks/tracksClass.py", line 241, in plot track.plot(plot_axis, chrom, start, end) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/pygenometracks/tracks/HiCMatrixTrack.py", line 245, in plot self.img = self.pcolormesh_45deg(ax, matrix, start_pos, vmax=vmax, vmin=vmin) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/pygenometracks/tracks/HiCMatrixTrack.py", line 314, in pcolormesh_45deg vmin=vmin, vmax=vmax, cmap=self.cmap, norm=self.norm) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/matplotlib/init.py", line 1412, in inner return func(ax, map(sanitize_sequence, args), kwargs) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/matplotlib/axes/_axes.py", line 6027, in pcolormesh collection._scale_norm(norm, vmin, vmax) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/matplotlib/cm.py", line 381, in _scale_norm "Passing parameters norm and vmin/vmax simultaneously is " ValueError: Passing parameters norm and vmin/vmax simultaneously is not supported. Please pass vmin/vmax directly to the norm when creating it.** (base) [shuangyang.wu@clip-login-0 merged]$ hicPlotTADs --tracks conf --region 1 --dpi 300 --outFileName Col_TADs.png --title "Col TADs on chr1" INFO:numexpr.utils:Note: detected 76 virtual cores but NumExpr set to maximum of 64, check "NUMEXPR_MAX_THREADS" environment variable. INFO:numexpr.utils:Note: NumExpr detected 76 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8. INFO:numexpr.utils:NumExpr defaulting to 8 threads.

title not set for 'section 2. [tads]' INFO:hicmatrix.HiCMatrix:Number of poor regions to remove: 92 {'1': 42, '2': 8, '3': 5, '4': 11, '5': 26} INFO:hicmatrix.HiCMatrix:found existing 92 nan bins that will be included for masking WARNING:hicmatrix.HiCMatrix:Bin size is not homogeneous. Median 40000

INFO:pygenometracks.tracksClass:time initializing track(s): INFO:pygenometracks.tracksClass:2.8513972759246826 Traceback (most recent call last): File "/users/shuangyang.wu/miniconda3/bin/hicPlotTADs", line 7, in main() File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/hicexplorer/hicPlotTADs.py", line 9, in main plotTracks.main(args) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/pygenometracks/plotTracks.py", line 306, in main region = get_region(args.region) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/pygenometracks/plotTracks.py", line 249, in get_region chrom, position = region_string.strip().split(":") ValueError: not enough values to unpack (expected 2, got 1)

joachimwolff commented 2 years ago

@lldelisle that bug happens in pyGenomeTracks. Any idea? Thanks!

lldelisle commented 2 years ago

Hi @wushyer, Can you give me more details so I can troubleshoot:

pgt --version
python --version
conda list | grep matplotlib
cat conf

Thanks

lldelisle commented 2 years ago

For the second bug:

$ hicPlotTADs --tracks conf --region 1 --dpi 300 --outFileName Col_TADs.png --title "Col TADs on chr1"

We don't support to give just the chromosome name. You need to provide chr:start-end.

wushyer commented 2 years ago

pgt 3.0

Python 3.7.10

matplotlib 3.5.0 pypi_0 pypi

cat conf

[hic] file = hic_Col_merged.40kb.Corrected.h5 title = Col chr1 colormap = RdYlBu_r depth = 300000 height = 7 transform = log1p file_type = hic_matrix

[tads] file = hic_Col_merged.40kb.Corrected_domains.bed file_type = domains border_color = black color = none line_width = 1.5 overlay_previous = share-y show_data_range = no

[x-axis] fontsize = 16 where = top

[tad score] file = hic_Col_merged.40kb.Corrected_tad_score.bm title = TAD separation score height = 4 file_type = bedgraph_matrix

[spacer]

[gene track] file = genes.bed height = 10 title = tair10 genes labels = false

lldelisle commented 2 years ago

This version of pgt is not compatible with this version of matplotlib.

wushyer commented 2 years ago

Hi which pgt shall I use then? Thanks

lldelisle commented 2 years ago

We still have not tested matplotlib 3.5.0. I think it should be compatible with the last version of pgt (3.6) but for sure it is not with 3.5.1: https://github.com/deeptools/pyGenomeTracks/commit/c7a4ee17ba30fcf6057abc33ef4d8d1be1f54717

lldelisle commented 2 years ago

In theory conda should not have allowed that. How did you trick conda? Did you installed pgt with pip?

lldelisle commented 2 years ago

(Sorry pgt is pyGenomeTracks which is behind hicPlotTADs)

wushyer commented 2 years ago

I failed the conda installation, I use the set.up install

lldelisle commented 2 years ago

I know it can be useful to have everything in your base environment but I think mixing conda with other type of installation may cause this type of bug. I personally use a different conda environment for HiCExplorer/pyGenomeTracks as they are strict on matplotlib versions. I do:

conda create -n hicexplorer -c bioconda -c conda-forge hicexplorer=3.7.2 python=3.7

Then when I want to use HiCExplorer:

conda activate hicexplorer
wushyer commented 2 years ago

pgt

Hi, I use this kind of installation but the network seems not so goog, that keep "solving environment". That's the reason I git the package with local installation. I will check this with the System Administrator. Thanks for the help

lldelisle commented 2 years ago

I now remember that I also had issues. I think the trick was to force numpy version:

conda create -n hicexplorer -c bioconda -c conda-forge hicexplorer=3.7.2 python=3.7 "numpy<1.20"
lldelisle commented 2 years ago

Maybe @joachimwolff would know this better than me.

wushyer commented 2 years ago

conda create -n hicexplorer -c bioconda -c conda-forge hicexplorer=3.7.2 python=3.7 "numpy<1.20"

(tools3) [shuangyang.wu@clip-login-0 Re-seq]$ conda create -n hicexplorer -c bioconda -c conda-forge hicexplorer=3.7.2 python=3.7 "numpy<1.20" Collecting package metadata (current_repodata.json): done Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source. Collecting package metadata (repodata.json): done Solving environment: -

It was also stuck in solve environment...

lldelisle commented 2 years ago

This worked for me (after a while but worked):

# create the environment using : `conda env create -f environment.yml`
# update existing environment using : `conda env update -f environment.yml`
# It installs from source using github maf branch.
name: hicexplorer3.7.2
channels:
   - bioconda
   - conda-forge
dependencies:
    - hicexplorer =3.7.2
    - numpy >=1.16
    - matplotlib >=3.1.1,<=3.2.2
    - intervaltree >=2.1.0
    - pybigwig >=0.3.16
    - future >=0.17.0
    - hicmatrix >=15
    - pysam >=0.14
    - pytest
    - gffutils >=0.9
    - pybedtools >=0.8.1
    - tqdm >=4.20
    - biopython >=1.69
    - python =3.8
    - pygenometracks =3.6
lldelisle commented 2 years ago

Also if you just need hicPlotTADs (pgt). This also work:

conda create -n pgt_3.6 -c bioconda -c conda-forge pygenometracks=3.6 python=3.7
wushyer commented 2 years ago

pgt --version

Hi, I install pgt 3.6 but with same error

(pgt_3.6) [shuangyang.wu@clip-login-1 merged]$ ~/miniconda3/bin/hicPlotTADs --tracks conf --region 1:100000-500000 --dpi 300 --outFileName Col_TADs.png --title "Col TADs on chr1" INFO:numexpr.utils:Note: detected 76 virtual cores but NumExpr set to maximum of 64, check "NUMEXPR_MAX_THREADS" environment variable. INFO:numexpr.utils:Note: NumExpr detected 76 cores but "NUMEXPR_MAX_THREADS" not set, so enforcing safe limit of 8. INFO:numexpr.utils:NumExpr defaulting to 8 threads.

title not set for 'section 2. [tads]' INFO:hicmatrix.HiCMatrix:Number of poor regions to remove: 92 {'1': 42, '2': 8, '3': 5, '4': 11, '5': 26} INFO:hicmatrix.HiCMatrix:found existing 92 nan bins that will be included for masking WARNING:hicmatrix.HiCMatrix:Bin size is not homogeneous. Median 40000

INFO:pygenometracks.tracksClass:time initializing track(s): INFO:pygenometracks.tracksClass:3.5763447284698486 DEBUG:pygenometracks.tracksClass:Figure size in cm is 40 x 23.6. Dpi is set to 300

INFO:pygenometracks.tracksClass:plotting 1. [hic] INFO:pygenometracks.tracks.GenomeTrack:setting min, max values for track 1. [hic] to: 84.37145272509095, 1146.9949544480594

Traceback (most recent call last): File "/users/shuangyang.wu/miniconda3/bin/hicPlotTADs", line 7, in main() File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/hicexplorer/hicPlotTADs.py", line 9, in main plotTracks.main(args) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/pygenometracks/plotTracks.py", line 307, in main trp.plot(args.outFileName, region, title=args.title) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/pygenometracks/tracksClass.py", line 241, in plot track.plot(plot_axis, chrom, start, end) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/pygenometracks/tracks/HiCMatrixTrack.py", line 245, in plot self.img = self.pcolormesh_45deg(ax, matrix, start_pos, vmax=vmax, vmin=vmin) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/pygenometracks/tracks/HiCMatrixTrack.py", line 314, in pcolormesh_45deg vmin=vmin, vmax=vmax, cmap=self.cmap, norm=self.norm) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/matplotlib/init.py", line 1412, in inner return func(ax, map(sanitize_sequence, args), **kwargs) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/matplotlib/axes/_axes.py", line 6027, in pcolormesh collection._scale_norm(norm, vmin, vmax) File "/users/shuangyang.wu/miniconda3/lib/python3.7/site-packages/matplotlib/cm.py", line 381, in _scale_norm "Passing parameters norm and vmin/vmax simultaneously is " ValueError: Passing parameters norm and vmin/vmax simultaneously is not supported. Please pass vmin/vmax directly to the norm when creating it. (pgt_3.6) [shuangyang.wu@clip-login-1 merged]$ pgt -version usage: pgt --tracks tracks.ini --region chr1:1000000-4000000 -o image.png pgt: error: the following arguments are required: --tracks, --outFileName/-out (pgt_3.6) [shuangyang.wu@clip-login-1 merged]$ pgt --version pgt 3.6

lldelisle commented 2 years ago

Can you get the version of matplotlib?

wushyer commented 2 years ago

conda list | grep matplotlib

matplotlib-base 3.3.2 py37h817c723_0

lldelisle commented 2 years ago

Wait. If you do:

~/miniconda3/bin/hicPlotTADs

Then you call the one you installed with python setup.py

lldelisle commented 2 years ago

You now need to do:

pgt --tracks conf --region 1:100000-500000 --dpi 300 --outFileName Col_TADs.png --title "Col TADs on chr1"
wushyer commented 2 years ago

~/miniconda3/bin/hicPlotTADs

(pgt_3.6) [shuangyang.wu@clip-login-1 merged]$ ~/miniconda3/bin/hicexplorer --version hicexplorer 2.2.1.1

Sorry, I forget the installation as I use multiple installation, pip, conda set.up, I think the bin path is the pip install as I never successfully install with conda....

lldelisle commented 2 years ago

Can you try now the line of code I put above which should use pgt version 3.6 and matplotlib 3.3.2?

pgt --tracks conf --region 1:100000-500000 --dpi 300 --outFileName Col_TADs.png --title "Col TADs on chr1"
wushyer commented 2 years ago

pgt --tracks conf --region 1:100000-500000 --dpi 300 --outFileName Col_TADs.png --title "Col TADs on chr1"

This works!!!!

wushyer commented 2 years ago

Can you try now the line of code I put above which should use pgt version 3.6 and matplotlib 3.3.2?

pgt --tracks conf --region 1:100000-500000 --dpi 300 --outFileName Col_TADs.png --title "Col TADs on chr1"

Thanks for all the kind help.

lldelisle commented 2 years ago

Ouf! Now I explain to you. If you create new conda environment and if you want to use what you just installed you need to give just the name of the software (pgt in this case) and it will use the software you just installed. If you put the full path like ~/miniconda3/bin/hicPlotTADs then it will use this specific software (in this case the version 3.0 of pgt) and in this case it used the matplotlib 3.5 you installed I don't know how.

lldelisle commented 2 years ago

Anyway bug solved. This is cool.