search

statgen / pheweb

A tool to build a website to browse hundreds or thousands of GWAS.
MIT License
154 stars 65 forks source link

Error 404s and missing LDs in WSGI based apache2 webserver #162

Closed eikematthias closed 3 years ago

eikematthias commented 3 years ago

Disclaimer: This is my first bug report and webserver.

Steps to reproduce: pheweb v. 1.3.9

config.py: hg_build_number=38 num_procs=4

  1. I prepared a locally running pheweb browser. All features work fine with pheweb serve --port 5000. Shut it down afterwards.

  2. Creating a wsgi token via pheweb wsgi

  3. Install apache2 and libapache2-mod-wsgi-py3 via apt install; sudo a2enmod wsgi

  4. Modify /etc/apache2/sites-available/000-default.conf to: <VirtualHost *:5000> ServerName localhost DocumentRoot /home/sukmb465

    WSGIScriptAlias / /home/sukmb465/Documents/pheweb38/generated-by-pheweb/wsgi.py
WSGIDaemonProcess pheweb user=sukmb465 group=sukmb465 processes=1 threads=1
WSGIProcessGroup pheweb

<Directory />
        Require all granted
</Directory>
LogLevel warn
ErrorLog ${APACHE_LOG_DIR}/pheweb_error.log
CustomLog ${APACHE_LOG_DIR}/pheweb_access.log combined

  5. make apache2 listen to port 5000 by adding in /etc/apache2/ports.conf: Listen 5000

  6. sudo systemctl restart apache2

  7. looking into localhost:5000 -> Phenotypes -> clicking on any 'Nearest Gene(s)' produces a Error 404. image

  8. LDs dont work at all

Thanks!

pjvandehaar commented 3 years ago

Can you post the apache error log?

On Tue, Mar 23, 2021 at 05:05 eikematthias @.***> wrote:

Disclaimer: This is my first bug report and webserver.

Steps to reproduce: pheweb v. 1.3.9

config.py: hg_build_number=38 num_procs=4

1.

I prepared a locally running pheweb browser. All features work fine with pheweb serve --port 500. Shut it down afterwards. 2.

Creating a wsgi token via pheweb wsgi 3.

Install apache2 and libapache2-mod-wsgi-py3 via apt install; sudo a2enmod wsgi 4.

Modify /etc/apache2/sites-available/000-default.conf to: <VirtualHost *:5000> ServerName localhost DocumentRoot /home/sukmb465

WSGIScriptAlias / /home/sukmb465/Documents/pheweb38/generated-by-pheweb/wsgi.py
WSGIDaemonProcess pheweb user=sukmb465 group=sukmb465 processes=1 threads=1
WSGIProcessGroup pheweb

<Directory />
        Require all granted
</Directory>
LogLevel warn
ErrorLog ${APACHE_LOG_DIR}/pheweb_error.log
CustomLog ${APACHE_LOG_DIR}/pheweb_access.log combined

1.

make apache2 listen to port 5000 by adding in /etc/apache2/ports.conf: Listen 5000 2.

sudo systemctl restart apache2 3.

looking into localhost:5000 -> Phenotypes -> clicking on any 'Nearest Gene(s)' produces a Error 404. [image: image] https://user-images.githubusercontent.com/71607460/112121017-0d1dc000-8bbf-11eb-8938-b7616e074b49.png 4.

LDs dont work at all

Thanks!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/statgen/pheweb/issues/162, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAGSPCLTGMIEGSW4HS3VJA3TFBKXBANCNFSM4ZUX76MQ .

eikematthias commented 3 years ago

I forgot to mention, this is on Ubuntu 20.10.

$ sudo cat /var/log/apache2/pheweb_error.log 
[Tue Mar 23 09:16:01.781131 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662] Filepath /home/sukmb465/Documents/pheweb38/generated-by-pheweb/resources/genes-v37-hg19.bed of kind genes was requested but doesn't exist
[Tue Mar 23 09:16:01.783219 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662] Traceback (most recent call last):
[Tue Mar 23 09:16:01.783388 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 305, in gene_phenocode_page
[Tue Mar 23 09:16:01.783418 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662]     gene_region_mapping = get_gene_region_mapping()
[Tue Mar 23 09:16:01.783449 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 288, in get_gene_region_mapping
[Tue Mar 23 09:16:01.783465 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662]     return {genename: (chrom, pos1, pos2) for chrom, pos1, pos2, genename in get_gene_tuples()}
[Tue Mar 23 09:16:01.783498 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 288, in <dictcomp>
[Tue Mar 23 09:16:01.783514 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662]     return {genename: (chrom, pos1, pos2) for chrom, pos1, pos2, genename in get_gene_tuples()}
[Tue Mar 23 09:16:01.783531 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/utils.py", line 98, in get_gene_tuples
[Tue Mar 23 09:16:01.783540 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662]     for chrom,start,end,genename,ensg in get_gene_tuples_with_ensg():
[Tue Mar 23 09:16:01.783556 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/utils.py", line 93, in get_gene_tuples_with_ensg
[Tue Mar 23 09:16:01.783564 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662]     with open(get_filepath('genes')) as f:
[Tue Mar 23 09:16:01.783579 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/file_utils.py", line 32, in get_filepath
[Tue Mar 23 09:16:01.783588 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662]     raise PheWebError("Filepath {} of kind {} was requested but doesn't exist".format(filepath, kind))
[Tue Mar 23 09:16:01.783658 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662] pheweb.utils.PheWebError: Filepath /home/sukmb465/Documents/pheweb38/generated-by-pheweb/resources/genes-v37-hg19.bed of kind genes was requested but doesn't exist
[Tue Mar 23 09:16:01.783705 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50662] Sorry, your region request for phenocode 'COVID19_HGI_B2_ALL_eur_leave_ukbb_23andme_20210107' and gene 'LZTFL1' didn't work
[Tue Mar 23 09:16:14.811023 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702] Filepath /home/sukmb465/Documents/pheweb38/generated-by-pheweb/resources/genes-v37-hg19.bed of kind genes was requested but doesn't exist
[Tue Mar 23 09:16:14.811732 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702] Traceback (most recent call last):
[Tue Mar 23 09:16:14.812001 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 305, in gene_phenocode_page
[Tue Mar 23 09:16:14.812056 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702]     gene_region_mapping = get_gene_region_mapping()
[Tue Mar 23 09:16:14.812146 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 288, in get_gene_region_mapping
[Tue Mar 23 09:16:14.812229 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702]     return {genename: (chrom, pos1, pos2) for chrom, pos1, pos2, genename in get_gene_tuples()}
[Tue Mar 23 09:16:14.812300 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 288, in <dictcomp>
[Tue Mar 23 09:16:14.812332 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702]     return {genename: (chrom, pos1, pos2) for chrom, pos1, pos2, genename in get_gene_tuples()}
[Tue Mar 23 09:16:14.812394 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/utils.py", line 98, in get_gene_tuples
[Tue Mar 23 09:16:14.812426 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702]     for chrom,start,end,genename,ensg in get_gene_tuples_with_ensg():
[Tue Mar 23 09:16:14.812487 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/utils.py", line 93, in get_gene_tuples_with_ensg
[Tue Mar 23 09:16:14.812518 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702]     with open(get_filepath('genes')) as f:
[Tue Mar 23 09:16:14.812575 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/file_utils.py", line 32, in get_filepath
[Tue Mar 23 09:16:14.812606 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702]     raise PheWebError("Filepath {} of kind {} was requested but doesn't exist".format(filepath, kind))
[Tue Mar 23 09:16:14.812726 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702] pheweb.utils.PheWebError: Filepath /home/sukmb465/Documents/pheweb38/generated-by-pheweb/resources/genes-v37-hg19.bed of kind genes was requested but doesn't exist
[Tue Mar 23 09:16:14.812851 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:50702] Sorry, your region request for phenocode 'COVID19_HGI_B2_ALL_leave_UKBB_23andme_20210107' and gene 'LZTFL1' didn't work
[Tue Mar 23 09:20:53.229502 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302] Filepath /home/sukmb465/Documents/pheweb38/generated-by-pheweb/resources/genes-v37-hg19.bed of kind genes was requested but doesn't exist
[Tue Mar 23 09:20:53.229670 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302] Traceback (most recent call last):
[Tue Mar 23 09:20:53.229721 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 305, in gene_phenocode_page
[Tue Mar 23 09:20:53.229728 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302]     gene_region_mapping = get_gene_region_mapping()
[Tue Mar 23 09:20:53.229738 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 288, in get_gene_region_mapping
[Tue Mar 23 09:20:53.229744 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302]     return {genename: (chrom, pos1, pos2) for chrom, pos1, pos2, genename in get_gene_tuples()}
[Tue Mar 23 09:20:53.229753 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 288, in <dictcomp>
[Tue Mar 23 09:20:53.229758 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302]     return {genename: (chrom, pos1, pos2) for chrom, pos1, pos2, genename in get_gene_tuples()}
[Tue Mar 23 09:20:53.229767 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/utils.py", line 98, in get_gene_tuples
[Tue Mar 23 09:20:53.229784 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302]     for chrom,start,end,genename,ensg in get_gene_tuples_with_ensg():
[Tue Mar 23 09:20:53.229793 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/utils.py", line 93, in get_gene_tuples_with_ensg
[Tue Mar 23 09:20:53.229798 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302]     with open(get_filepath('genes')) as f:
[Tue Mar 23 09:20:53.229808 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/file_utils.py", line 32, in get_filepath
[Tue Mar 23 09:20:53.229815 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302]     raise PheWebError("Filepath {} of kind {} was requested but doesn't exist".format(filepath, kind))
[Tue Mar 23 09:20:53.229847 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302] pheweb.utils.PheWebError: Filepath /home/sukmb465/Documents/pheweb38/generated-by-pheweb/resources/genes-v37-hg19.bed of kind genes was requested but doesn't exist
[Tue Mar 23 09:20:53.229874 2021] [wsgi:error] [pid 9198:tid 140202652747328] [remote 127.0.0.1:51302] Sorry, your region request for phenocode 'COVID19_HGI_B2_ALL_leave_23andme_20210107' and gene 'LZTFL1' didn't work
Exception ignored in: <generator object path at 0x7f8370b4ff90>
Traceback (most recent call last):
  File "/usr/lib/python3.8/importlib/resources.py", line 190, in path
NameError: name 'FileNotFoundError' is not defined
[Tue Mar 23 09:54:45.582005 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820] Filepath /home/sukmb465/Documents/pheweb38/generated-by-pheweb/resources/genes-v37-hg19.bed of kind genes was requested but doesn't exist
[Tue Mar 23 09:54:45.582633 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820] Traceback (most recent call last):
[Tue Mar 23 09:54:45.582681 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 305, in gene_phenocode_page
[Tue Mar 23 09:54:45.582688 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820]     gene_region_mapping = get_gene_region_mapping()
[Tue Mar 23 09:54:45.582698 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 288, in get_gene_region_mapping
[Tue Mar 23 09:54:45.582703 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820]     return {genename: (chrom, pos1, pos2) for chrom, pos1, pos2, genename in get_gene_tuples()}
[Tue Mar 23 09:54:45.582712 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 288, in <dictcomp>
[Tue Mar 23 09:54:45.582717 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820]     return {genename: (chrom, pos1, pos2) for chrom, pos1, pos2, genename in get_gene_tuples()}
[Tue Mar 23 09:54:45.582725 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/utils.py", line 98, in get_gene_tuples
[Tue Mar 23 09:54:45.582730 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820]     for chrom,start,end,genename,ensg in get_gene_tuples_with_ensg():
[Tue Mar 23 09:54:45.582738 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/utils.py", line 93, in get_gene_tuples_with_ensg
[Tue Mar 23 09:54:45.582743 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820]     with open(get_filepath('genes')) as f:
[Tue Mar 23 09:54:45.582751 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/file_utils.py", line 32, in get_filepath
[Tue Mar 23 09:54:45.582773 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820]     raise PheWebError("Filepath {} of kind {} was requested but doesn't exist".format(filepath, kind))
[Tue Mar 23 09:54:45.582800 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820] pheweb.utils.PheWebError: Filepath /home/sukmb465/Documents/pheweb38/generated-by-pheweb/resources/genes-v37-hg19.bed of kind genes was requested but doesn't exist
[Tue Mar 23 09:54:45.582818 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:55820] Sorry, your region request for phenocode 'COVID19_HGI_B2_ALL_leave_UKBB_23andme_20210107' and gene 'LZTFL1' didn't work
[Tue Mar 23 10:02:43.801093 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908] Filepath /home/sukmb465/Documents/pheweb38/generated-by-pheweb/resources/genes-v37-hg19.bed of kind genes was requested but doesn't exist
[Tue Mar 23 10:02:43.802092 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908] Traceback (most recent call last):
[Tue Mar 23 10:02:43.802497 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 305, in gene_phenocode_page
[Tue Mar 23 10:02:43.802559 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908]     gene_region_mapping = get_gene_region_mapping()
[Tue Mar 23 10:02:43.802655 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 288, in get_gene_region_mapping
[Tue Mar 23 10:02:43.802696 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908]     return {genename: (chrom, pos1, pos2) for chrom, pos1, pos2, genename in get_gene_tuples()}
[Tue Mar 23 10:02:43.802787 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 288, in <dictcomp>
[Tue Mar 23 10:02:43.802829 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908]     return {genename: (chrom, pos1, pos2) for chrom, pos1, pos2, genename in get_gene_tuples()}
[Tue Mar 23 10:02:43.802980 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/utils.py", line 98, in get_gene_tuples
[Tue Mar 23 10:02:43.803030 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908]     for chrom,start,end,genename,ensg in get_gene_tuples_with_ensg():
[Tue Mar 23 10:02:43.803100 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/utils.py", line 93, in get_gene_tuples_with_ensg
[Tue Mar 23 10:02:43.803136 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908]     with open(get_filepath('genes')) as f:
[Tue Mar 23 10:02:43.803196 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/file_utils.py", line 32, in get_filepath
[Tue Mar 23 10:02:43.803227 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908]     raise PheWebError("Filepath {} of kind {} was requested but doesn't exist".format(filepath, kind))
[Tue Mar 23 10:02:43.803378 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908] pheweb.utils.PheWebError: Filepath /home/sukmb465/Documents/pheweb38/generated-by-pheweb/resources/genes-v37-hg19.bed of kind genes was requested but doesn't exist
[Tue Mar 23 10:02:43.803510 2021] [wsgi:error] [pid 10137:tid 139978764101184] [remote 127.0.0.1:56908] Sorry, your region request for phenocode 'COVID19_HGI_B2_ALL_leave_23andme_20210107' and gene 'LZTFL1' didn't work
Exception ignored in: <generator object path at 0x7f4f5006ef90>
Traceback (most recent call last):
  File "/usr/lib/python3.8/importlib/resources.py", line 190, in path
NameError: name 'FileNotFoundError' is not defined
[Wed Mar 24 08:16:15.302547 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512] Filepath /home/sukmb465/Documents/pheweb38/generated-by-pheweb/resources/genes-v37-hg19.bed of kind genes was requested but doesn't exist
[Wed Mar 24 08:16:15.304015 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512] Traceback (most recent call last):
[Wed Mar 24 08:16:15.304107 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 305, in gene_phenocode_page
[Wed Mar 24 08:16:15.304123 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512]     gene_region_mapping = get_gene_region_mapping()
[Wed Mar 24 08:16:15.304140 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 288, in get_gene_region_mapping
[Wed Mar 24 08:16:15.304148 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512]     return {genename: (chrom, pos1, pos2) for chrom, pos1, pos2, genename in get_gene_tuples()}
[Wed Mar 24 08:16:15.304162 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 288, in <dictcomp>
[Wed Mar 24 08:16:15.304171 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512]     return {genename: (chrom, pos1, pos2) for chrom, pos1, pos2, genename in get_gene_tuples()}
[Wed Mar 24 08:16:15.304188 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/utils.py", line 98, in get_gene_tuples
[Wed Mar 24 08:16:15.304196 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512]     for chrom,start,end,genename,ensg in get_gene_tuples_with_ensg():
[Wed Mar 24 08:16:15.304226 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/utils.py", line 93, in get_gene_tuples_with_ensg
[Wed Mar 24 08:16:15.304236 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512]     with open(get_filepath('genes')) as f:
[Wed Mar 24 08:16:15.304247 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512]   File "/home/sukmb465/.local/lib/python3.8/site-packages/pheweb/file_utils.py", line 32, in get_filepath
[Wed Mar 24 08:16:15.304253 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512]     raise PheWebError("Filepath {} of kind {} was requested but doesn't exist".format(filepath, kind))
[Wed Mar 24 08:16:15.304286 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512] pheweb.utils.PheWebError: Filepath /home/sukmb465/Documents/pheweb38/generated-by-pheweb/resources/genes-v37-hg19.bed of kind genes was requested but doesn't exist
[Wed Mar 24 08:16:15.304316 2021] [wsgi:error] [pid 5548:tid 139812463175232] [remote 127.0.0.1:54512] Sorry, your region request for phenocode 'COVID19_HGI_B2_ALL_leave_UKBB_23andme_20210107' and gene 'LZTFL1' didn't work
pjvandehaar commented 3 years ago

Oh, I see. Python in your wsgi.py file isn’t loading config.py, so it doesn’t know you’re on build 38. config.py used to get loaded implicitly, but I changed it so the commandline-handling and child-spawning step loads config.py explicitly. wsgi skips the commandline, so it misses config.py. And I never use wsgi so I didn't notice.

You've got two options:

Option 1: Instead of using wsgi, just run pheweb serve --host=localhost --port=5000 and then tell apache2 to reverse proxy to that. Example apache2 config is at https://github.com/statgen/pheweb/blob/master/etc/detailed-apache2-instructions/pheweb.conf . pheweb serve can be managed by systemd like https://github.com/statgen/pheweb/blob/master/etc/pheweb.service .

Option 2: Add these lines to your wsgi.py:

import pheweb.conf
pheweb.conf.load_overrides_from_file("/home/sukmb465/Documents/pheweb38/config.py")

I'd like to know if those lines work to fix your bug (so I can update https://github.com/statgen/pheweb/blob/master/pheweb/load/wsgi.py), but I recommend using pheweb serve.

eikematthias commented 3 years ago

Thank you for your help! Option 2 works, fixing both issues I had! Though a new Issue came up: The search does not find any matches except for phenotypes. Any queries will be answered with 500 Internal Server Error.

My webserver-machine gives following log with: sudo cat /var/log/apache2/pheweb_error.log:

[Thu Mar 25 12:38:56.539957 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266] [2021-03-25 12:38:56,539] ERROR in app: Exception on /go [GET]
[Thu Mar 25 12:38:56.540025 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266] Traceback (most recent call last):
[Thu Mar 25 12:38:56.540037 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]   File "/home/pheweb/.local/lib/python3.8/site-packages/flask/app.py", line 2447, in wsgi_app
[Thu Mar 25 12:38:56.540047 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]     response = self.full_dispatch_request()
[Thu Mar 25 12:38:56.540056 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]   File "/home/pheweb/.local/lib/python3.8/site-packages/flask/app.py", line 1952, in full_dispatch_request
[Thu Mar 25 12:38:56.540066 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]     rv = self.handle_user_exception(e)
[Thu Mar 25 12:38:56.540074 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]   File "/home/pheweb/.local/lib/python3.8/site-packages/flask/app.py", line 1821, in handle_user_exception
[Thu Mar 25 12:38:56.540084 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]     reraise(exc_type, exc_value, tb)
[Thu Mar 25 12:38:56.540092 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]   File "/home/pheweb/.local/lib/python3.8/site-packages/flask/_compat.py", line 39, in reraise
[Thu Mar 25 12:38:56.540101 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]     raise value
[Thu Mar 25 12:38:56.540110 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]   File "/home/pheweb/.local/lib/python3.8/site-packages/flask/app.py", line 1950, in full_dispatch_request
[Thu Mar 25 12:38:56.540119 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]     rv = self.dispatch_request()
[Thu Mar 25 12:38:56.540127 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]   File "/home/pheweb/.local/lib/python3.8/site-packages/flask/app.py", line 1936, in dispatch_request
[Thu Mar 25 12:38:56.540159 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]     return self.view_functions[rule.endpoint](**req.view_args)
[Thu Mar 25 12:38:56.540168 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]   File "/home/pheweb/.local/lib/python3.8/site-packages/pheweb/serve/server.py", line 91, in go
[Thu Mar 25 12:38:56.540177 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]     best_suggestion = autocompleter.get_best_completion(query)
[Thu Mar 25 12:38:56.540186 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]   File "/home/pheweb/.local/lib/python3.8/site-packages/pheweb/serve/autocomplete.py", line 52, in get_best_completion
[Thu Mar 25 12:38:56.540218 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]     suggestions = self.autocomplete(query)
[Thu Mar 25 12:38:56.540227 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]   File "/home/pheweb/.local/lib/python3.8/site-packages/pheweb/serve/autocomplete.py", line 46, in autocomplete
[Thu Mar 25 12:38:56.540236 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]     result = list(itertools.islice(autocompleter(query), 0, 10))
[Thu Mar 25 12:38:56.540245 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]   File "/home/pheweb/.local/lib/python3.8/site-packages/pheweb/serve/autocomplete.py", line 107, in _autocomplete_rsid
[Thu Mar 25 12:38:56.540254 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266]     rsid_cpra_pairs = list(self._cpras_rsids_sqlite3.execute(
[Thu Mar 25 12:38:56.540263 2021] [wsgi:error] [pid 32291:tid 140551524824832] [remote 134.245.228.102:60266] sqlite3.ProgrammingError: SQLite objects created in a thread can only be used in that same thread. The object was created in thread id 140551558395648 and this is thread id 140551524824832.

Option1 was my bandaid-approach in the meanwhile, though the search box did not work completely, so you always had to wait for the autocompletion to be done to select the correct gene/SNP, if not and you hit impatiently enter after your search input, the clients browser starts to look at your clients localhost:5000, with, of course, no answer to be expected.

pjvandehaar commented 3 years ago

I just released version 1.3.12 which fixes the sqlite3 threading issue (I think– I haven't tested it threaded) and uses relative URLs for the 302 redirects (so that you won't get localhost no matter what hostname a server thinks it has).

I'm looking into why the autocomplete is so slow.

pjvandehaar commented 3 years ago

Disclaimer: This is my first bug report and webserver.

Thanks for all the detailed descriptions. You've been really helpful, and it's great to have these issues solved.

pjvandehaar commented 3 years ago

Autocomplete is fast in version 1.3.13. Thanks for getting me to finally fix that. 👍

You can re-open this issue if it's not actually fixed, but I'm closing it for now.