Closed Guasupito closed 5 months ago
Dear Guillermo,
The sole purpose of the televir-server component is to install databases and software relevant for the viral detection module.
After finishing, it is not needed anymore. so it is normal to not see a televir-server component.
As to a solution:
I would suggest to begin by re-running up.sh
if, as seems to be indicated by your last log, your docker installation is older than 5 months. This will ensure that an updated version of TELEVIR is installed.
i. From what I could see the databases seem to have been installed correctly (the cp error reported does not impact installation). To check, just access the web application. On the Homepage, Projects and Settings pages, you should see 3 icons, namely a new section for the viral detection module.
ii. If you still don't seem to have the viral detection functionality, you can try, on your terminal:
docker exec -it insaflu-server restart-apache
.iii. If then you still don't have the funcionality, from your log I'm seeing one possible issue, it should be as simple to solve as renaming a filename. Just go inside the insaflu-server instance:
docker exec -it insaflu-server bash
Check the directory /televir/mngs_benchmark/ for the file utility_docker.db. e.g.
ls /televir/mngs_benchmark/utility_docker.db
If not, and another version of this file is present (e.g. utility_docker-test.db), then please rename it and restart apache and you should be good to go. e.g.
mv /televir/mngs_benchmark/utility_docker-test.db /televir/mngs_benchmark/utility_docker.db
Finally, if no utility_docker.db file or variant is present in this directory, then please let us know for further investigation.
In principle, no extra configurations are needed for deployment behind a proxy. At least with regards to Televir Software and Database installation, as attested by their successful installation (from the logs).
Please let us know how this goes, At your disposal, On behalf of the INSaFLU team, Joao
Good morning Joao
We are trying for several weeks to make the new install work.
Finally, editing the scripts of the televir and insaflu installation in orther to use our proxy, we were able to install correctly without errors the televir module. We added this line at the begining of each .sh script in orther to everythin could work trhough our proxy:
set -e export http_proxy=http://proxy.san.gva.es:8080
But now, when we run the up.sh script, all seams to be running fine, except apache. This is the execution:
administrador@gaspra:/insaflu/docker$ ./up.sh
version
is obsolete
[+] Running 10/10
✔ insaflu-server Pulled 3.4s
✔ 2d473b07cdd5 Already exists 0.0s
✔ 991df59af78b Already exists 0.0s
✔ c1d4295e0db5 Already exists 0.0s
✔ fac14ea36bb6 Already exists 0.0s
✔ 78f019514494 Already exists 0.0s
✔ 09d7808d3eaf Already exists 0.0s
✔ 35c9b0dc3a2d Already exists 0.0s
✔ c734c0654d5f Already exists 0.0s
✔ f53d813ea84d Already exists 0.0s
[+] Running 4/0
✔ Container postgres Running 0.0s
✔ Container docker-officer-1 Running 0.0s
✔ Container postgres_backup Running 0.0s
✔ Container 448c8c8eabda_insaflu-server Created 0.0s
Attaching to 448c8c8eabda_insaflu-server
448c8c8eabda_insaflu-server | ---> Wait 45 seconds for all pgsql services ...
448c8c8eabda_insaflu-server | ---> Collect static data ...
448c8c8eabda_insaflu-server | /insaflu_web/INSaFLU
448c8c8eabda_insaflu-server |
448c8c8eabda_insaflu-server | 0 static files copied to '/insaflu_web/INSaFLU/static_all', 1082 unmodified.
448c8c8eabda_insaflu-server | ---> Create/Update database if necessary ...
448c8c8eabda_insaflu-server | /insaflu_web/INSaFLU
448c8c8eabda_insaflu-server | Operations to perform:
448c8c8eabda_insaflu-server | Apply all migrations: admin, auth, contenttypes, datasets, extend_user, log_login, manage_virus, managing_files, pathogen_identification, sessions, settings
448c8c8eabda_insaflu-server | Running migrations:
448c8c8eabda_insaflu-server | No migrations to apply.
448c8c8eabda_insaflu-server | ---> Load default files ...
448c8c8eabda_insaflu-server | /insaflu_web/INSaFLU
448c8c8eabda_insaflu-server | Upload abricate files
448c8c8eabda_insaflu-server | Define default database fields
448c8c8eabda_insaflu-server | Set default users...
448c8c8eabda_insaflu-server | Upload References
448c8c8eabda_insaflu-server | #Upload references: 0
448c8c8eabda_insaflu-server | Set species tag to all references
448c8c8eabda_insaflu-server | Number of references processed on species tag: 91
448c8c8eabda_insaflu-server | Test bed and gff files for all references
448c8c8eabda_insaflu-server | Number of references processed: 91
448c8c8eabda_insaflu-server | End
448c8c8eabda_insaflu-server | /insaflu_web/INSaFLU
448c8c8eabda_insaflu-server | Replace old technology names...
448c8c8eabda_insaflu-server | Set default pipelines..
448c8c8eabda_insaflu-server | Set obsolete softwares because of parameters...
448c8c8eabda_insaflu-server | Set pipelines in previous softwares...
448c8c8eabda_insaflu-server | Refresh software names...
448c8c8eabda_insaflu-server | End
448c8c8eabda_insaflu-server | /insaflu_web/INSaFLU
448c8c8eabda_insaflu-server | Starting update pangolin
448c8c8eabda_insaflu-server | Success update pangolin
448c8c8eabda_insaflu-server | ---> Set owners flu_user ...
448c8c8eabda_insaflu-server | ---> Start sge ...
448c8c8eabda_insaflu-server | Starting Grid Engine qmaster
448c8c8eabda_insaflu-server | Starting Grid Engine execution daemon
448c8c8eabda_insaflu-server | /insaflu_web/INSaFLU
448c8c8eabda_insaflu-server | ---> Start apache server ...
448c8c8eabda_insaflu-server | httpd not running, trying to start
448c8c8eabda_insaflu-server | ---> apache running ...The script stops here, and after many many minutes we got this error: 448c8c8eabda_insaflu-server exited with code 137 and the container is stoped.
What could be happening?
We also tried to uninstall everything, including docker, and start again. But always stop in the same place.
Thanks for your help.
Dear Guillermo,
It is normal that the script stops there, as it means the website should be running.
If you go to the browser to localhost:8080 (or the expected IP) INSaFLU should be running.
If not, please see eg. #22 #26 to see what may be causing it
Basically, this usually fixes it for me (assuming the correct IP is already in ALLOWED_HOSTS):
ctrl^c
chmod -R 777 $BASE_PATH_DATA/insaflu/log [where $BASE_PATH_DATA is the folder you defined in .env]
.> /up.sh
If you want the service to run in the background, instead of up.sh just do:
docker compose up insaflu-server -d
Hope it helps, Daniel
Good afternoon Daniel
After our last communication we followed your recommendations, but after starting the insaflu-server we lost access to the web. We tried many thinks, but nothing worked, so we decided to move the server to another location where de proxy isn't needed and reinstall the server.
We use Ubuntu 20.04 LTS as the Operating System (we encounter problems with more modern versions, even we tried to install Rocky, but the server is a bit old and didn't boot).
With Ubuntu 20.04 we were able to install Insaflu-server using docker following the instructions, but we cannot deploy televir-server.
The image is generated, but when is about to start the container and is executing the entripoint.sh script it fails:
administrador@gaspra:/insaflu/docker$ sudo ./up_televir.sh
version
is obsolete
[+] Running 1/1
! televir-server Warning pull access denied for douradojns, repository does not exist or may require 'docker login': denied: requested access to the res... 1.4s
[+] Building 13.8s (37/37) FINISHED docker:default
=> [televir-server internal] load build definition from Dockerfile 0.3s
=> => transferring dockerfile: 2.33kB 0.0s
=> [televir-server internal] load metadata for docker.io/library/ubuntu:latest 1.4s
=> [televir-server internal] load .dockerignore 0.3s
=> => transferring context: 2B 0.0s
=> [televir-server 26/32] ADD https://api.github.com/repos/SantosJGND/TELEVIR/git/refs/heads/main version.json 0.3s
=> [televir-server 1/32] FROM docker.io/library/ubuntu:latest@sha256:3f85b7caad41a95462cf5b787d8a04604c8262cdcdf9a472b8c52ef83375fe15 0.0s
=> [televir-server internal] load build context 0.3s
=> => transferring context: 979B 0.0s
=> CACHED [televir-server 2/32] RUN useradd -ms /bin/bash flu_user 0.0s
=> CACHED [televir-server 3/32] RUN set -xe && apt-get update && apt-get install -y python3 python3-pip python3-venv 0.0s
=> CACHED [televir-server 4/32] RUN python3 -m venv /opt/venv 0.0s
=> CACHED [televir-server 5/32] RUN pip install --upgrade pip 0.0s
=> CACHED [televir-server 6/32] RUN apt-get update 0.0s
=> CACHED [televir-server 7/32] RUN apt-get install -y build-essential wget curl rsync apt-utils python3-setuptools 0.0s
=> CACHED [televir-server 8/32] RUN apt-get clean 0.0s
=> CACHED [televir-server 9/32] RUN rm -rf /var/lib/apt/lists/* 0.0s
=> CACHED [televir-server 10/32] RUN pip install --upgrade pip 0.0s
=> CACHED [televir-server 11/32] RUN apt-get update && apt-get install -y gcc python3-dev musl-dev 0.0s
=> CACHED [televir-server 12/32] RUN apt-get install -y git ncbi-entrez-direct tabix samtools bioperl 0.0s
=> CACHED [televir-server 13/32] RUN apt-get -y install default-jre python3-pip 0.0s
=> CACHED [televir-server 14/32] RUN wget --quiet https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh -O ~/miniconda.sh && /bin/bash ~/miniconda.sh -b -p /o 0.0s
=> CACHED [televir-server 15/32] RUN wget https://github.com/pachterlab/kallisto/releases/download/v0.43.1/kallisto_linux-v0.43.1.tar.gz 0.0s
=> CACHED [televir-server 16/32] RUN tar -xzf kallisto_linux-v0.43.1.tar.gz 0.0s
=> CACHED [televir-server 17/32] RUN apt-get install zlib1g-dev make g++ 0.0s
=> CACHED [televir-server 18/32] RUN python -m venv /opt/venv 0.0s
=> CACHED [televir-server 19/32] RUN /opt/venv/bin/pip install --upgrade pip 0.0s
=> CACHED [televir-server 20/32] RUN apt-get install -y postgresql postgresql-contrib 0.0s
=> CACHED [televir-server 21/32] RUN apt-get install libpq-dev python3-dev 0.0s
=> CACHED [televir-server 22/32] RUN /opt/venv/bin/pip install setuptools==57.4.0 numpy pandas psycopg2==2.9.6 sqlalchemy python-decouple==3.8 danio xopen==1.7.0 fastq_filter==0. 0.0s
=> CACHED [televir-server 23/32] RUN /opt/venv/bin/pip install --default-timeout=100 future 0.0s
=> CACHED [televir-server 24/32] COPY taxdump.tar.gz /opt/ 0.0s
=> CACHED [televir-server 25/32] WORKDIR /insaflu_web 0.0s
=> CACHED [televir-server 26/32] ADD https://api.github.com/repos/SantosJGND/TELEVIR/git/refs/heads/main version.json 0.0s
=> CACHED [televir-server 27/32] RUN wget --quiet https://github.com/SantosJGND/TELEVIR/archive/main.zip -O televir.zip && unzip -q televir.zip && rm televir.zip && m 0.0s
=> CACHED [televir-server 28/32] COPY configs/televir.env /insaflu_web/TELEVIR/.env 0.0s
=> [televir-server 29/32] COPY entrypoint.sh entrypoint_original.sh 1.2s
=> [televir-server 30/32] RUN sed "s/APP_USER/flu_user/g" entrypoint_original.sh > entrypoint.sh; rm entrypoint_original.sh 2.7s
=> [televir-server 31/32] RUN chmod a+x entrypoint.sh 3.1s
=> [televir-server] exporting to image 2.3s
=> => exporting layers 1.7s
=> => writing image sha256:e5417a9c709432cba8285d28e0c1a73e1f65f0b7495802d337a370d34b0af71c 0.1s
=> => naming to docker.io/library/douradojns:televir-image 0.1s
[+] Running 4/4
✔ Container postgres Running 0.0s
✔ Container docker-officer-1 Running 0.0s
✔ Container postgres_backup Running 0.0s
✔ Container televir-server Created 2.0s
Attaching to televir-server
televir-server | ---> Create televir dbs in /televir/mngs_benchmark/ ...
televir-server | Traceback (most recent call last):
televir-server | File "/insaflu_web/TELEVIR/main.py", line 214, in I have been working on this for several days and I am out of ideas about what could be happening.
Any thoughts?
Dear Guillermo,
First of all, Sorry for the delay in replying.
And thank you for this report. We have found it to have been caused by a software version update we had not controlled for.
I am happy to say that this issue has been addressed.
To run the latest version of the code, please travel to your INSaFLU docker directory and pull the latest version of this git repository. after that, re-run up.sh and up_televir.sh, in that order. i.e.
cd docker/
git pull origin master
./up.sh
./up_televir.sh
The installation will take a while.
Thank you again, Please do not hesitate in contacting us should you encounter any other problem, Best regards, Joao
Good morning,
Finally all is up and working!
After following your instructions I had to connect to the insaflu-server and delete the utility_docker.db and re-run the up_televir.sh script in order to be able to run the televir module.
But now it seams is all working.
We will install the server in it's final possition and check if it works fine behind the proxy server.
Kind regards.
Good morning Daniel
We are trying to add the viral detection module to our local installation of INSAFLU at FISABIO, but after executing the up_televir.sh script, it finish with a code 0 and this message after a few seconds execution:
root@gaspra:~/docker# ./up_televir.sh
And after that, if we check the docker we don't see a televir-server component, instead we see this:
root@gaspra:~/docker# docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES b3e3ebf3b8f8 funkyfuture/deck-chores:1 "deck-chores" 29 seconds ago Up 24 seconds docker-officer-1 d93d1110829a monsantopinheiro:insaflu-server "/entrypoint.sh init…" 10 months ago Up 5 months 0.0.0.0:8080->80/tcp, :::8080->80/tcp insaflu-server 48a022de7f6e prodrigestivill/postgres-backup-local "/bin/sh -c 'exec /u…" 10 months ago Up 5 months (healthy) 5432/tcp postgres_backup 30a2d45fc014 docker-db_insaflu "docker-entrypoint.s…" 10 months ago Up 5 months 5432/tcp postgres
It seams that a container named docker-officer-1 was generated after executing the script.
INSAFLU has to be stoped before executing the up_televir.sh script?
We are behind a proxy server. We need to configure anithing extra in orther to the script to download all its data correctly?
Thanks foryour help and kind regards, Guillermo.