bioinfoUGR / sRNAtoolbox

small RNA analysis programs: standalone jar files (sRNAbench, sRNAde, etc), manuals and Docker image
MIT License
10 stars 3 forks source link

Can't populate the database #3

Open David-Caceres opened 2 years ago

David-Caceres commented 2 years ago

Hi

I'm using sRNAbench in docker and populate option doesn't work. It seems a certificate problem. I received a "insecurequestwarning" error (urllib3). I tried to disable these warnings with python with no result at all.

cris12gm commented 2 years ago

Dear @David-Caceres, thanks for your feedback.

I think the error can be caused by an old version of sRNAtoolbox docker as this bug should be solved in the last version. Try to pull the last one from dockerhub: docker pull ugrbioinfo/srnatoolbox:0.0.5 or docker pull ugrbioinfo/srnatoolbox:latest

Let us know if with that it is solved or if you have any more problems.

Best, Cristina

David-Caceres commented 2 years ago

Hi again

Same result, but the new version shows a more extended error:

srna@sRNAtoolbox:~$ populate -db /opt/sRNAtoolboxDB/ Traceback (most recent call last): File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connection.py", line 157, in _new_conn (self._dns_host, self.port), self.timeout, **extra_kw File "/home/srna/.local/lib/python3.6/site-packages/urllib3/util/connection.py", line 61, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen chunked=chunked, File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 376, in _make_request self._validate_conn(conn) File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn conn.connect() File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connection.py", line 300, in connect conn = self._new_conn() File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connection.py", line 169, in _new_conn self, "Failed to establish a new connection: %s" % e urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7f35109085f8>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/srna/.local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 720, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/home/srna/.local/lib/python3.6/site-packages/urllib3/util/retry.py", line 436, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='bioinfo2.ugr.es', port=443): Max retries exceeded with url: /sRNAtoolboxDB/populate.php (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f35109085f8>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/srv/bin/populate", line 107, in showspecies=url_consulta("showspecies","true",php) File "/srv/bin/populate", line 13, in url_consulta response = (requests.post(web, data,verify=False)).text File "/home/srna/.local/lib/python3.6/site-packages/requests/api.py", line 116, in post return request('post', url, data=data, json=json, kwargs) File "/home/srna/.local/lib/python3.6/site-packages/requests/api.py", line 60, in request return session.request(method=method, url=url, kwargs) File "/home/srna/.local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request resp = self.send(prep, send_kwargs) File "/home/srna/.local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send r = adapter.send(request, kwargs) File "/home/srna/.local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='bioinfo2.ugr.es', port=443): Max retries exceeded with url: /sRNAtoolboxDB/populate.php (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f35109085f8>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))

Thanks

David

David-Caceres commented 2 years ago

Ok, it worked in my laptop now, ubutu distro, not in my other machine (Debian).

But some of the genomes I requested are empty.

srna@sRNAtoolbox:~$ cat info

Installed genome assemblies and annotations: -- Note that the names of these files have to be used with the following parameters: species=for example species=hg38_prim_mp (the bowtie index of the genome assembly) libs=for example libs=hg38_prim_ncRNA (for the profiling of other ncRNAs)

################################################

Scientific name // Assembly // Bowtie Index (species=)

Homo sapiens // GRCh38_p13 // GRCh38_p13_mp

######### #########

Libs(libs=) // Description

GRCh38_p13_cdna.fa GRCh38_p13_genomic_tRNA.fa GRCh38_p13_ncRNA.fa GRCh38_p13_RNAcentral.fa

################################################

Scientific name // Assembly // Bowtie Index (species=)

Bacteria collection // bacterial_collection // bacterial_collection

######### #########

Libs(libs=) // Description

################################################

Scientific name // Assembly // Bowtie Index (species=)

Human virus collection // human_tagged_virus // human_tagged_virus

######### #########

Libs(libs=) // Description

David-Caceres commented 2 years ago

After giving writting permissions to the landing directories, it is finally working fine.

Much appreciated.

David

cris12gm commented 2 years ago

Dear David,

Thanks a lot for the update on the problem, I'm glad you were able to finally get it solved.

Best, Cristina

criphg commented 1 year ago

I also have a problem with populate in docker image even following last advises:

Traceback (most recent call last): File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connection.py", line 157, in _new_conn (self._dns_host, self.port), self.timeout, **extra_kw File "/home/srna/.local/lib/python3.6/site-packages/urllib3/util/connection.py", line 61, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen chunked=chunked, File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 376, in _make_request self._validate_conn(conn) File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn conn.connect() File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connection.py", line 300, in connect conn = self._new_conn() File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connection.py", line 169, in _new_conn self, "Failed to establish a new connection: %s" % e urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7ffadd25b550>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/srna/.local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 720, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/home/srna/.local/lib/python3.6/site-packages/urllib3/util/retry.py", line 436, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='bioinfo2.ugr.es', port=443): Max retries exceeded with url: /sRNAtoolboxDB/populate.php (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ffadd25b550>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/srv/bin/populate", line 107, in showspecies=url_consulta("showspecies","true",php) File "/srv/bin/populate", line 13, in url_consulta response = (requests.post(web, data,verify=False)).text File "/home/srna/.local/lib/python3.6/site-packages/requests/api.py", line 116, in post return request('post', url, data=data, json=json, kwargs) File "/home/srna/.local/lib/python3.6/site-packages/requests/api.py", line 60, in request return session.request(method=method, url=url, kwargs) File "/home/srna/.local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request resp = self.send(prep, send_kwargs) File "/home/srna/.local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send r = adapter.send(request, kwargs) File "/home/srna/.local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='bioinfo2.ugr.es', port=443): Max retries exceeded with url: /sRNAtoolboxDB/populate.php (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ffadd25b550>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))

Could you help me?

David-Caceres commented 1 year ago

I also have a problem with populate in docker image even following last advises:

Traceback (most recent call last): File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connection.py", line 157, in _new_conn (self._dns_host, self.port), self.timeout, **extra_kw File "/home/srna/.local/lib/python3.6/site-packages/urllib3/util/connection.py", line 61, in create_connection for res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM): File "/usr/lib/python3.6/socket.py", line 745, in getaddrinfo for res in _socket.getaddrinfo(host, port, family, type, proto, flags): socket.gaierror: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 672, in urlopen chunked=chunked, File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 376, in _make_request self._validate_conn(conn) File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 994, in _validate_conn conn.connect() File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connection.py", line 300, in connect conn = self._new_conn() File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connection.py", line 169, in _new_conn self, "Failed to establish a new connection: %s" % e urllib3.exceptions.NewConnectionError: <urllib3.connection.VerifiedHTTPSConnection object at 0x7ffadd25b550>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/srna/.local/lib/python3.6/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/home/srna/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 720, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/home/srna/.local/lib/python3.6/site-packages/urllib3/util/retry.py", line 436, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='bioinfo2.ugr.es', port=443): Max retries exceeded with url: /sRNAtoolboxDB/populate.php (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ffadd25b550>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/srv/bin/populate", line 107, in showspecies=url_consulta("showspecies","true",php) File "/srv/bin/populate", line 13, in url_consulta response = (requests.post(web, data,verify=False)).text File "/home/srna/.local/lib/python3.6/site-packages/requests/api.py", line 116, in post return request('post', url, data=data, json=json, kwargs) File "/home/srna/.local/lib/python3.6/site-packages/requests/api.py", line 60, in request return session.request(method=method, url=url, kwargs) File "/home/srna/.local/lib/python3.6/site-packages/requests/sessions.py", line 533, in request resp = self.send(prep, send_kwargs) File "/home/srna/.local/lib/python3.6/site-packages/requests/sessions.py", line 646, in send r = adapter.send(request, kwargs) File "/home/srna/.local/lib/python3.6/site-packages/requests/adapters.py", line 516, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='bioinfo2.ugr.es', port=443): Max retries exceeded with url: /sRNAtoolboxDB/populate.php (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7ffadd25b550>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution',))

Could you help me?

Hi,

I solved it installing the last docker version (see https://github.com/bioinfoUGR/sRNAtoolbox/issues/3#issuecomment-1196753143) and giving write and execute permissions to other users in landng directories.

Hope it helps

David

bioinfoUGR commented 1 year ago

Dear @criphg ,

Sorry for the inconvenience. As mentioned by David, the problem should be solved if you download the last version from dockerhub (https://hub.docker.com/r/ugrbioinfo/srnatoolbox). We just updated it so everything matches our recently published book chapter (https://link.springer.com/protocol/10.1007/978-1-0716-2982-6_13). We also experienced some network problems in our servers which maybe was the cause of your problem, and should be fixed now. Let us know if the problem is solved with any of these tips.

Best, Cristina