search

theislab / sfaira

data and model repository for single-cell data
https://sfaira.readthedocs.io
BSD 3-Clause "New" or "Revised" License
133 stars 13 forks source link

Read timed out #692

Closed Cem-Gulec closed 1 year ago

Cem-Gulec commented 1 year ago

Describe the bug I tried with different machines and connections to see whether my internet was the problem causing this but none of them worked so far. I currently am going through the tutorials for "cellxgene_download" but I get ReadTimeoutError.

To Reproduce I haven't tried it with conda environment yet but this behavior happens on usual pip installment. Below is the snippet I use:

from sfaira.data.dataloaders.databases import DatasetSuperGroupDatabases
import os

cache_path = os.path.join(".", "data")
dsg = DatasetSuperGroupDatabases(data_path=cache_path)
target_collections = [x.collection_id for x in dsg.dataset_groups][0]
dsg.subset(key="collection_id", values=target_collections)
dsg.download()
dsg.load()
adatas = dsg.adata_ls

sfaira version: 0.3.12

Expected behavior To successfully acquire dsg as in the Line 5.

System [please complete the following information]:

Additional context How error is traced back:

* Owlready2 * Warning: optimized Cython parser module 'owlready2_optimized' is not available, defaulting to slower Python implementation
Matplotlib is building the font cache; this may take a moment.
2022-09-20 20:00:08.082917: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations:  AVX2 FMA
To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
2022-09-20 20:00:08.241977: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libcudart.so.11.0'; dlerror: libcudart.so.11.0: cannot open shared object file: No such file or directory
2022-09-20 20:00:08.242224: I tensorflow/stream_executor/cuda/cudart_stub.cc:29] Ignore above cudart dlerror if you do not have a GPU set up on your machine.
2022-09-20 20:00:08.282054: E tensorflow/stream_executor/cuda/cuda_blas.cc:2981] Unable to register cuBLAS factory: Attempting to register factory for plugin cuBLAS when one has already been registered
2022-09-20 20:00:09.169826: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer.so.7'; dlerror: libnvinfer.so.7: cannot open shared object file: No such file or directory
2022-09-20 20:00:09.170192: W tensorflow/stream_executor/platform/default/dso_loader.cc:64] Could not load dynamic library 'libnvinfer_plugin.so.7'; dlerror: libnvinfer_plugin.so.7: cannot open shared object file: No such file or directory
2022-09-20 20:00:09.170311: W tensorflow/compiler/tf2tensorrt/utils/py_utils.cc:38] TF-TRT Warning: Cannot dlopen some TensorRT libraries. If you would like to use Nvidia GPU with TensorRT, please make sure the missing libraries mentioned above are installed properly.
Downloading: ncbitaxon_v2021-06-10.obo to /root/.cache/sfaira/ontologies/ncbitaxon
Downloading: efo.obo to /root/.cache/sfaira/ontologies/efo
Downloading: hsapdv_master.obo to /root/.cache/sfaira/ontologies/hsapdv
Downloading: mmusdv.obo to /root/.cache/sfaira/ontologies/mmusdv
Downloading: uberon_v2021-07-27.obo to /root/.cache/sfaira/ontologies/uberon
Ontology <class 'sfaira.versions.metadata.base.OntologyUberonLifecyclestage'> is not a DAG, treat child-parent reasoning with care.
Downloading: mondo_v2021-08-11.obo to /root/.cache/sfaira/ontologies/mondo
Ontology <class 'sfaira.versions.metadata.base.OntologyMondo'> is not a DAG, treat child-parent reasoning with care.
Ontology <class 'sfaira.versions.metadata.base.OntologyUberon'> is not a DAG, treat child-parent reasoning with care.
Downloading: pato_v2021-08-06.obo to /root/.cache/sfaira/ontologies/pato
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 445, in _make_request
    six.raise_from(e, None)
  File "<string>", line 3, in raise_from
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 440, in _make_request
    httplib_response = conn.getresponse()
  File "/usr/lib/python3.9/http/client.py", line 1347, in getresponse
    response.begin()
  File "/usr/lib/python3.9/http/client.py", line 307, in begin
    version, status, reason = self._read_status()
  File "/usr/lib/python3.9/http/client.py", line 268, in _read_status
    line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
  File "/usr/lib/python3.9/socket.py", line 704, in readinto
    return self._sock.recv_into(b)
  File "/usr/lib/python3.9/ssl.py", line 1241, in recv_into
    return self.read(nbytes, buffer)
  File "/usr/lib/python3.9/ssl.py", line 1099, in read
    return self._sslobj.read(len, buffer)
socket.timeout: The read operation timed out

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 699, in urlopen
    httplib_response = self._make_request(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 447, in _make_request
    self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 336, in _raise_timeout
    raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='api.cellxgene.cziscience.com', port=443): Read timed out. (read timeout=5)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 439, in send
    resp = conn.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 783, in urlopen
    return self.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 783, in urlopen
    return self.urlopen(
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 783, in urlopen
    return self.urlopen(
  [Previous line repeated 2 more times]
  File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 755, in urlopen
    retries = retries.increment(
  File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 574, in increment
    raise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='api.cellxgene.cziscience.com', port=443): Max retries exceeded with url: /dp/v1/collections/a238e9fa-2bdf-41df-8522-69046f99baff (Caused by ReadTimeoutError("HTTPSConnectionPool(host='api.cellxgene.cziscience.com', port=443): Read timed out. (read timeout=5)"))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/cguelec/playground.py", line 5, in <module>
    dsg = DatasetSuperGroupDatabases(data_path=cache_path)
  File "/usr/local/lib/python3.9/dist-packages/sfaira/data/dataloaders/databases/super_group.py", line 17, in _init_
    DatasetSuperGroupCellxgene(
  File "/usr/local/lib/python3.9/dist-packages/sfaira/data/dataloaders/databases/cellxgene/cellxgene_group.py", line 83, in _init_
    dataset_groups = [
  File "/usr/local/lib/python3.9/dist-packages/sfaira/data/dataloaders/databases/cellxgene/cellxgene_group.py", line 84, in <listcomp>
    DatasetGroupCellxgene(
  File "/usr/local/lib/python3.9/dist-packages/sfaira/data/dataloaders/databases/cellxgene/cellxgene_group.py", line 28, in _init_
    dataset_ids = [x["id"] for x in get_collection(collection_id=collection_id)['datasets']]
  File "/usr/local/lib/python3.9/dist-packages/sfaira/data/dataloaders/databases/cellxgene/rest_helpers.py", line 58, in get_collection
    return rest_api_collection_request(url=CELLXGENE_PRODUCTION_ENDPOINT + COLLECTIONS + collection_id)
  File "/usr/local/lib/python3.9/dist-packages/sfaira/data/dataloaders/databases/cellxgene/rest_helpers.py", line 38, in rest_api_collection_request
    r = https.get(url)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 555, in get
    return self.request('GET', url, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 542, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python3/dist-packages/requests/sessions.py", line 655, in send
    r = adapter.send(request, **kwargs)
  File "/usr/local/lib/python3.9/dist-packages/sfaira/data/dataloaders/databases/cellxgene/rest_helpers.py", line 25, in send
    return super().send(request, **kwargs)
  File "/usr/lib/python3/dist-packages/requests/adapters.py", line 516, in send
    raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api.cellxgene.cziscience.com', port=443): Max retries exceeded with url: /dp/v1/collections/a238e9fa-2bdf-41df-8522-69046f99baff (Caused by ReadTimeoutError("HTTPSConnectionPool(host='api.cellxgene.cziscience.com', port=443): Read timed out. (read timeout=5)"))