search

huwenboshi / hess

Estimate local SNP heritability and genetic covariance from GWAS summary association statistics.
http://bogdan.bioinformatics.ucla.edu/software/hess/
GNU General Public License v3.0
40 stars 11 forks source link

ERROR Missing files for the reference panel #21

Closed heihaheihaha closed 1 year ago

heihaheihaha commented 1 year ago

I tried STEP1-compute eigenvalues, squared projections, and product of projections in my computer, and get the ERROR：

[ERROR] Missing files for the reference panel /home/ubuntu/hess-0.5.3-beta/bfile/1kg_eur_1pct/1kg_eur_1pct_chr8.bed I have tried other commands to view this file, they all worked, there is no problem with the path I specified. I wonder if it is because I specified the wrong file(not the bed file?) or the program does not have sufficient permissions to access this file? I attach the command I tried to execute below： for chrom in $(seq 22) do python /home/ubuntu/hess-0.5.3-beta/hess.py \ --local-rhog /home/ubuntu/mdd_dd/dd/dd.tsv /home/ubuntu/mdd_dd/mdd/mdd.tsv \ --chrom $chrom \ --partition /home/ubuntu/hess-0.5.3-beta/partition/nygcresearch-ldetect-data-ac125e47bf7f/EUR/fourier_ls-chr$chrom.bed \ --bfile /home/ubuntu/hess-0.5.3-beta/bfile/1kg_eur_1pct/1kg_eur_1pct_chr$chrom.bed \ --out /home/ubuntu/mdd_dd/hess_out/step1 done Full output（CHR22as example）： [INFO] @----------------------------------------------------------@ HESS v0.5 9/October/2017
(C) 2017 Huwenbo Shi, GNU General Public License, v3
----------------------------------------------------------
For documentation, citation & bug-report instructions:
http://bogdan.bioinformatics.ucla.edu/software/hess/

@----------------------------------------------------------@ [INFO] Command started at: Sun, 22 Jan 2023 18:36:08 [INFO] Command issued: /home/ubuntu/hess-0.5.3-beta/hess.py \ --bfile /home/ubuntu/hess-0.5.3-beta/bfile/1kg_eur_1pct/1kg_eur_1pct_chr22.bed \ --local-rhog /home/ubuntu/mdd_dd/dd/dd.tsv /home/ubuntu/mdd_dd/mdd/mdd.tsv \ --partition /home/ubuntu/hess-0.5.3-beta/partition/nygcresearch-ldetect-data-ac125e47bf7f/EUR/fourier_ls-chr22.bed \ --out /home/ubuntu/mdd_dd/hess_out/step1 \ --chrom 22 /home/ubuntu/hess-0.5.3-beta/src/refpanel.py:13: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'. partition = pd.read_table(filename, delim_whitespace=True) [INFO] Loaded 24 partitions on chromosome 22 [INFO] Average window size is 1466370 [ERROR] Missing files for the reference panel /home/ubuntu/hess-0.5.3-beta/bfile/1kg_eur_1pct/1kg_eur_1pct_chr22.bed

Environment info: /# packages in environment at /home/ubuntu/miniconda3/envs/hess: /# /# Name Version Build Channel _libgcc_mutex 0.1 conda_forge conda-forge _openmp_mutex 4.5 2_gnu conda-forge backports 1.0 pyhd8ed1ab_3 conda-forge backports.functools_lru_cache 1.6.1 py_0 conda-forge backports_abc 0.5 py_1 conda-forge ca-certificates 2022.12.7 ha878542_0 conda-forge certifi 2019.11.28 py27h8c360ce_1 conda-forge cycler 0.10.0 py_2 conda-forge dbus 1.13.6 hfdff14a_1 conda-forge expat 2.5.0 h27087fc_0 conda-forge fontconfig 2.14.1 hc2a2eb6_0 conda-forge freetype 2.12.1 hca18f0e_1 conda-forge functools32 3.2.3.2 py_3 conda-forge futures 3.3.0 py27h8c360ce_1 conda-forge gettext 0.21.1 h27087fc_0 conda-forge glib 2.66.3 h58526e2_0 conda-forge gst-plugins-base 1.14.5 h0935bb2_2 conda-forge gstreamer 1.14.5 h36ae1b5_2 conda-forge icu 64.2 he1b5a44_1 conda-forge jpeg 9e h166bdaf_2 conda-forge kiwisolver 1.1.0 py27h9e3301b_1 conda-forge ld_impl_linux-64 2.39 hcc3a1bd_1 conda-forge libblas 3.9.0 8_openblas conda-forge libcblas 3.9.0 8_openblas conda-forge libffi 3.2.1 he1b5a44_1007 conda-forge libgcc-ng 12.2.0 h65d4601_19 conda-forge libgfortran-ng 7.5.0 h14aa051_20 conda-forge libgfortran4 7.5.0 h14aa051_20 conda-forge libglib 2.66.3 hbe7bbb4_0 conda-forge libgomp 12.2.0 h65d4601_19 conda-forge libiconv 1.17 h166bdaf_0 conda-forge liblapack 3.9.0 8_openblas conda-forge libopenblas 0.3.12 pthreads_hb3c22a3_1 conda-forge libpng 1.6.39 h753d276_0 conda-forge libsqlite 3.40.0 h753d276_0 conda-forge libstdcxx-ng 12.2.0 h46fd767_19 conda-forge libuuid 2.32.1 h7f98852_1000 conda-forge libxcb 1.13 h7f98852_1004 conda-forge libxml2 2.9.10 hee79883_0 conda-forge libzlib 1.2.13 h166bdaf_4 conda-forge matplotlib 2.2.5 ha770c72_3 conda-forge matplotlib-base 2.2.5 py27h250f245_1 conda-forge ncurses 6.3 h27087fc_1 conda-forge numpy 1.16.5 py27h95a1406_0 conda-forge openssl 1.1.1s h0b41bf4_1 conda-forge pandas 0.24.2 py27hb3f55d8_0 conda-forge pcre 8.45 h9c3ff4c_0 conda-forge pip 20.1.1 pyh9f0ad1d_0 conda-forge pthread-stubs 0.4 h36c2ea0_1001 conda-forge pyparsing 2.4.7 pyh9f0ad1d_0 conda-forge pyqt 5.9.2 py27hcca6a23_4 conda-forge pysnptools 0.3.13 py27h95a95ce_6 bioconda python 2.7.15 h5a48372_1011_cpython conda-forge python-dateutil 2.8.1 py_0 conda-forge python_abi 2.7 1_cp27mu conda-forge pytz 2020.1 pyh9f0ad1d_0 conda-forge qt 5.9.7 h0c104cb_3 conda-forge readline 8.1.2 h0f457ee_0 conda-forge scipy 1.2.1 py27h921218d_2 conda-forge setuptools 44.0.0 py27_0 conda-forge singledispatch 3.6.1 pyh44b312d_0 conda-forge sip 4.19.8 py27hf484d3e_1000 conda-forge six 1.16.0 pyh6c4a22f_0 conda-forge sqlite 3.40.0 h4ff8645_0 conda-forge subprocess32 3.5.4 py27h516909a_0 conda-forge tk 8.6.12 h27826a3_0 conda-forge tornado 5.1.1 py27h14c3975_1000 conda-forge wheel 0.37.1 pyhd8ed1ab_0 conda-forge xorg-libxau 1.0.9 h7f98852_0 conda-forge xorg-libxdmcp 1.1.3 h7f98852_0 conda-forge xz 5.2.6 h166bdaf_0 conda-forge zlib 1.2.13 h166bdaf_4 conda-forge

umbrellaling commented 1 year ago

Have you solved this problem? I had the same problem and couldn't find a solution.

heihaheihaha commented 1 year ago

Have you solved this problem? I had the same problem and couldn't find a solution.

Yes, I sloving the problem but that was about a long time ago. I think the error may caused by Syntax error. Maybe you can check you command to make sure that the $chrom is corrected interpretd by shell. If you still have the problem, can you show your code?

Zhangzzzzzy commented 1 year ago

Have you solved this problem? I had the same problem and couldn't find a solution.

Yes, I sloving the problem but that was about a long time ago. I think the error may caused by Syntax error. Maybe you can check you command to make sure that the $chrom is corrected interpretd by shell. If you still have the problem, can you show your code?

Hi, I also met your kind of problem, my code is as follows： for chrom in $(seq 22) do python hess.py \ --local-rhog /data/BMI.txt /data/DEPn.txt \ --chrom $chrom \ --bfile /1kg_eur_1pct/1kg_eur_1pct_chr$chrom.bed \ --partition fourier_ls-all.bed \ --out result1/step1 done Error reported: /home/zhang/pycharm-community-2023.2.1/packages/hess/src/refpanel.py:13: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'. partition = pd.read_table(filename, delim_whitespace=True) [INFO] Loaded 24 partitions on chromosome 21 [INFO] Average window size is 1612854 [ERROR] Missing files for the reference panel /1kg_eur_1pct/1kg_eur_1pct_chr21.bed [INFO] @----------------------------------------------------------@ HESS v0.5 9/October/2017
(C) 2017 Huwenbo Shi, GNU General Public License, v3
----------------------------------------------------------
For documentation, citation & bug-report instructions:
http://bogdan.bioinformatics.ucla.edu/software/hess/ 
   @----------------------------------------------------------@

[INFO] Command started at: Tue, 31 Oct 2023 16:28:06 [INFO] Command issued: hess.py \ --bfile /1kg_eur_1pct/1kg_eur_1pct_chr22.bed \ --local-rhog /data/BMI.txt /data/DEPn.txt \ --partition fourier_ls-all.bed \ --out result1/step1 \ --chrom 22 /home/zhang/pycharm-community-2023.2.1/packages/hess/src/refpanel.py:13: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'. partition = pd.read_table(filename, delim_whitespace=True) [INFO] Loaded 24 partitions on chromosome 22 [INFO] Average window size is 1466370 [ERROR] Missing files for the reference panel /1kg_eur_1pct/1kg_eur_1pct_chr22.bed. Every chromosome has this error, is there an error anywhere?

heihaheihaha commented 1 year ago

Have you solved this problem? I had the same problem and couldn't find a solution.

Yes, I sloving the problem but that was about a long time ago. I think the error may caused by Syntax error. Maybe you can check you command to make sure that the $chrom is corrected interpretd by shell. If you still have the problem, can you show your code?

Hi, I also met your kind of problem, my code is as follows： for chrom in $(seq 22) do python hess.py --local-rhog /data/BMI.txt /data/DEPn.txt --chrom $chrom --bfile /1kg_eur_1pct/1kg_eur_1pct_chr$chrom.bed --partition fourier_ls-all.bed --out result1/step1 done Error reported: /home/zhang/pycharm-community-2023.2.1/packages/hess/src/refpanel.py:13: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'. partition = pd.read_table(filename, delim_whitespace=True) [INFO] Loaded 24 partitions on chromosome 21 [INFO] Average window size is 1612854 [ERROR] Missing files for the reference panel /1kg_eur_1pct/1kg_eur_1pct_chr21.bed [INFO] @----------------------------------------------------------@ | HESS | v0.5 | 9/October/2017 | |----------------------------------------------------------| | (C) 2017 Huwenbo Shi, GNU General Public License, v3 | |----------------------------------------------------------| | For documentation, citation & bug-report instructions: | | http://bogdan.bioinformatics.ucla.edu/software/hess/ | @----------------------------------------------------------@ [INFO] Command started at: Tue, 31 Oct 2023 16:28:06 [INFO] Command issued: hess.py --bfile /1kg_eur_1pct/1kg_eur_1pct_chr22.bed --local-rhog /data/BMI.txt /data/DEPn.txt --partition fourier_ls-all.bed --out result1/step1 --chrom 22 /home/zhang/pycharm-community-2023.2.1/packages/hess/src/refpanel.py:13: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'. partition = pd.read_table(filename, delim_whitespace=True) [INFO] Loaded 24 partitions on chromosome 22 [INFO] Average window size is 1466370 [ERROR] Missing files for the reference panel /1kg_eur_1pct/1kg_eur_1pct_chr22.bed. Every chromosome has this error, is there an error anywhere?

You file path seems a little puzzling, do you put all your file under /?

you can try $head /1kg_eur_1pct/1kg_eur_1pct_chr22.bed
or something else to make sure 1kg_eur_1pct_chr22.bed is truly there.

Zhangzzzzzy commented 1 year ago

Have you solved this problem? I had the same problem and couldn't find a solution.

Yes, I sloving the problem but that was about a long time ago. I think the error may caused by Syntax error. Maybe you can check you command to make sure that the $chrom is corrected interpretd by shell. If you still have the problem, can you show your code?

Hi, I also met your kind of problem, my code is as follows： for chrom in $(seq 22) do python hess.py --local-rhog /data/BMI.txt /data/DEPn.txt --chrom $chrom --bfile /1kg_eur_1pct/1kg_eur_1pct_chr$chrom.bed --partition fourier_ls-all.bed --out result1/step1 done Error reported: /home/zhang/pycharm-community-2023.2.1/packages/hess/src/refpanel.py:13: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'. partition = pd.read_table(filename, delim_whitespace=True) [INFO] Loaded 24 partitions on chromosome 21 [INFO] Average window size is 1612854 [ERROR] Missing files for the reference panel /1kg_eur_1pct/1kg_eur_1pct_chr21.bed [INFO] @----------------------------------------------------------@ | HESS | v0.5 | 9/October/2017 | |----------------------------------------------------------| | (C) 2017 Huwenbo Shi, GNU General Public License, v3 | |----------------------------------------------------------| | For documentation, citation & bug-report instructions: | | http://bogdan.bioinformatics.ucla.edu/software/hess/ | @----------------------------------------------------------@ [INFO] Command started at: Tue, 31 Oct 2023 16:28:06 [INFO] Command issued: hess.py --bfile /1kg_eur_1pct/1kg_eur_1pct_chr22.bed --local-rhog /data/BMI.txt /data/DEPn.txt --partition fourier_ls-all.bed --out result1/step1 --chrom 22 /home/zhang/pycharm-community-2023.2.1/packages/hess/src/refpanel.py:13: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'. partition = pd.read_table(filename, delim_whitespace=True) [INFO] Loaded 24 partitions on chromosome 22 [INFO] Average window size is 1466370 [ERROR] Missing files for the reference panel /1kg_eur_1pct/1kg_eur_1pct_chr22.bed. Every chromosome has this error, is there an error anywhere?

You file path seems a little puzzling, do you put all your file under /?

you can try $head /1kg_eur_1pct/1kg_eur_1pct_chr22.bed or something else to make sure 1kg_eur_1pct_chr22.bed is truly there.

In fact, I initially thought it was a path issue as well, and I changed the path a few times. Now I found the solution, I incorrectly added the .bed to the panel. thank you very much for your reply. Of course the reported paths shown to you were also faulty, now I have changed them all and it worked, thanks again!

Zhangzzzzzy commented 1 year ago

Have you solved this problem? I had the same problem and couldn't find a solution.

Yes, I sloving the problem but that was about a long time ago. I think the error may caused by Syntax error. Maybe you can check you command to make sure that the $chrom is corrected interpretd by shell. If you still have the problem, can you show your code?

Hi, I also met your kind of problem, my code is as follows： for chrom in $(seq 22) do python hess.py --local-rhog /data/BMI.txt /data/DEPn.txt --chrom $chrom --bfile /1kg_eur_1pct/1kg_eur_1pct_chr$chrom.bed --partition fourier_ls-all.bed --out result1/step1 done Error reported: /home/zhang/pycharm-community-2023.2.1/packages/hess/src/refpanel.py:13: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'. partition = pd.read_table(filename, delim_whitespace=True) [INFO] Loaded 24 partitions on chromosome 21 [INFO] Average window size is 1612854 [ERROR] Missing files for the reference panel /1kg_eur_1pct/1kg_eur_1pct_chr21.bed [INFO] @----------------------------------------------------------@ | HESS | v0.5 | 9/October/2017 | |----------------------------------------------------------| | (C) 2017 Huwenbo Shi, GNU General Public License, v3 | |----------------------------------------------------------| | For documentation, citation & bug-report instructions: | | http://bogdan.bioinformatics.ucla.edu/software/hess/ | @----------------------------------------------------------@ [INFO] Command started at: Tue, 31 Oct 2023 16:28:06 [INFO] Command issued: hess.py --bfile /1kg_eur_1pct/1kg_eur_1pct_chr22.bed --local-rhog /data/BMI.txt /data/DEPn.txt --partition fourier_ls-all.bed --out result1/step1 --chrom 22 /home/zhang/pycharm-community-2023.2.1/packages/hess/src/refpanel.py:13: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'. partition = pd.read_table(filename, delim_whitespace=True) [INFO] Loaded 24 partitions on chromosome 22 [INFO] Average window size is 1466370 [ERROR] Missing files for the reference panel /1kg_eur_1pct/1kg_eur_1pct_chr22.bed. Every chromosome has this error, is there an error anywhere?

You file path seems a little puzzling, do you put all your file under /?

you can try $head /1kg_eur_1pct/1kg_eur_1pct_chr22.bed or something else to make sure 1kg_eur_1pct_chr22.bed is truly there.

HELLO, I met a new question and I think the only way to get an answer here is to ask it at your place. Here is the code I had to get and the errors I got from running it：

python hess.py \ --local-rhog data/bmim.txt data/depm.txt \ --chrom 1 \ --bfile 1kg_eur_1pct/1kg_eur_1pct_chr1 \ --partition ldetect-data/fourier_ls-all.bed \ --out test/test1/step1 [INFO] @----------------------------------------------------------@ HESS v0.5 9/October/2017
(C) 2017 Huwenbo Shi, GNU General Public License, v3
----------------------------------------------------------
For documentation, citation & bug-report instructions:
http://bogdan.bioinformatics.ucla.edu/software/hess/ 
   @----------------------------------------------------------@

[INFO] Command started at: Tue, 31 Oct 2023 19:28:25 [INFO] Command issued: hess.py \ --bfile 1kg_eur_1pct/1kg_eur_1pct_chr1 \ --local-rhog data/bmim.txt data/depm.txt \ --partition ldetect-data/fourier_ls-all.bed \ --out test/test1/step1 \ --chrom 1 /home/zhang/pycharm-community-2023.2.1/packages/hess/src/refpanel.py:13: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'. partition = pd.read_table(filename, delim_whitespace=True) [INFO] Loaded 133 partitions on chromosome 1 [INFO] Average window size is 1873901 /home/zhang/pycharm-community-2023.2.1/packages/hess/src/refpanel.py:61: FutureWarning: read_table is deprecated, use read_csv instead, passing sep='\t'. names=['SNP', 'BP', 'A0', 'A1']) [INFO] 670954 SNPs read from reference panel Traceback (most recent call last): File "hess.py", line 217, in main() File "hess.py", line 45, in main argmap['out']) File "/home/zhang/pycharm-community-2023.2.1/packages/hess/src/estimation.py", line 389, in local_rhog_step1 sumstats1 = SumStats(sumstats_fnm[0], chrom) File "/home/zhang/pycharm-community-2023.2.1/packages/hess/src/sumstats.py", line 113, in init idx = idx_map[name]; val = cols[idx] IndexError: list index out of range。

I'm not sure what this message means: names=['SNP', 'BP', 'A0', 'A1'], and we don't have the column A0 in our summary data. In addition, at the end of the message there is an error: IndexError: list index out of range. what does this mean? Looking forward to your reply!