search

chaidiscovery / chai-lab

Chai-1, SOTA model for biomolecular structure prediction
https://www.chaidiscovery.com
Other
1.03k stars 132 forks source link

Cannot download conformers.apkl #35

Closed kimdn closed 1 week ago

kimdn commented 1 week ago

I just ran python examples/predict_structure.py

Then, I see

Traceback (most recent call last):
  File "/people/kimd999/bin/Miniconda3-py311_24.5.0-0-Linux-x86_64/lib/python3.11/site-packages/urllib3/connection.py", line 196, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/people/kimd999/bin/Miniconda3-py311_24.5.0-0-Linux-x86_64/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/people/kimd999/bin/Miniconda3-py311_24.5.0-0-Linux-x86_64/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/people/kimd999/bin/Miniconda3-py311_24.5.0-0-Linux-x86_64/lib/python3.11/site-packages/urllib3/connectionpool.py", line 789, in urlopen
    response = self._make_request(
               ^^^^^^^^^^^^^^^^^^^
  File "/people/kimd999/bin/Miniconda3-py311_24.5.0-0-Linux-x86_64/lib/python3.11/site-packages/urllib3/connectionpool.py", line 490, in _make_request
    raise new_e
  File "/people/kimd999/bin/Miniconda3-py311_24.5.0-0-Linux-x86_64/lib/python3.11/site-packages/urllib3/connectionpool.py", line 466, in _make_request
    self._validate_conn(conn)
  File "/people/kimd999/bin/Miniconda3-py311_24.5.0-0-Linux-x86_64/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1095, in _validate_conn
    conn.connect()
  File "/people/kimd999/bin/Miniconda3-py311_24.5.0-0-Linux-x86_64/lib/python3.11/site-packages/urllib3/connection.py", line 615, in connect
    self.sock = sock = self._new_conn()
                       ^^^^^^^^^^^^^^^^
  File "/people/kimd999/bin/Miniconda3-py311_24.5.0-0-Linux-x86_64/lib/python3.11/site-packages/urllib3/connection.py", line 211, in _new_conn
    raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x2ba7d5276a90>: Failed to establish a new connection: [Errno 101] Network is unreachable

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/people/kimd999/bin/Miniconda3-py311_24.5.0-0-Linux-x86_64/lib/python3.11/site-packages/requests/adapters.py", line 589, in send
    resp = conn.urlopen(
           ^^^^^^^^^^^^^
  File "/people/kimd999/bin/Miniconda3-py311_24.5.0-0-Linux-x86_64/lib/python3.11/site-packages/urllib3/connectionpool.py", line 843, in urlopen
    retries = retries.increment(
              ^^^^^^^^^^^^^^^^^^
  File "/people/kimd999/bin/Miniconda3-py311_24.5.0-0-Linux-x86_64/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment
    raise MaxRetryError(_pool, url, reason) from reason  # type: ignore[arg-type]
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='chaiassets.com', port=443): Max retries exceeded with url: /chai1-inference-depencencies/conformers.apkl (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x2ba7d5276a90>: Failed to establish a new connection: [Errno 101] Network is unreachable'))

It appears that both chaiassets.com and its IP (04.21.70.252) are not accessible both by web browser. Screenshot 2024-09-12 at 1 12 07 PM Screenshot 2024-09-12 at 1 14 38 PM

However, I can ping it well as

Screenshot 2024-09-12 at 1 14 34 PM

jackdent commented 1 week ago

Hi @kimdn, the URL you are trying to reach looks accessible to me: https://chaiassets.com/chai1-inference-depencencies/conformers.apkl

Is it possible something is misconfigured in your networking stack?

kimdn commented 1 week ago

Hm, I can access (even downloaded) https://chaiassets.com/chai1-inference-depencencies/conformers.apkl

However, still I can't access https://chaiassets.com

jackdent commented 1 week ago

Hi @kimdn, https://chaiassets.com is supposed to return an "Error 404", since we don't host anything at that URL. If you can download the URL I linked above (and also download https://chaiassets.com/chai1-inference-depencencies/models/256/trunk.pt2), things should be working for you fine.

jackdent commented 1 week ago

The error you posted above says that script is failing to download the file I linked in my previous comment, so if you can download it fine from your browser, I would guess that there's some issue in your Python setup:

urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='chaiassets.com', port=443): Max retries exceeded with url: /chai1-inference-depencencies/conformers.apkl (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x2ba7d5276a90>: Failed to establish a new connection: [Errno 101] Network is unreachable'))

kimdn commented 1 week ago

Thank you for your comments.

At my linux login node, I could download by wget https://chaiassets.com/chai1-inference-depencencies/models/256/trunk.pt2 . wget https://chaiassets.com/chai1-inference-depencencies/conformers.apkl .

I moved these files into downloads and downloads/models folder as I see https://github.com/chaidiscovery/chai-lab/blob/472d59f9914ea568f2863ff5c3172f4cac841d47/chai_lab/utils/paths.py#L45

Unfortunately, still I see same errors.

downloading https://chaiassets.com/chai1-inference-depencencies/conformers.apkl
Traceback (most recent call last):
  File "/people/kimd999/bin/Miniconda3-py3.12.4_Linux/lib/python3.12/site-packages/urllib3/connection.py", line 196, in _new_conn
    sock = connection.create_connection(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/people/kimd999/bin/Miniconda3-py3.12.4_Linux/lib/python3.12/site-packages/urllib3/util/connection.py", line 85, in create_connection
    raise err
  File "/people/kimd999/bin/Miniconda3-py3.12.4_Linux/lib/python3.12/site-packages/urllib3/util/connection.py", line 73, in create_connection
    sock.connect(sa)
OSError: [Errno 101] Network is unreachable

Weird part is that even when I commented out chai_lab/utils/paths.py's 

'''
@dataclasses.dataclass
class Downloadable:
    url: str
    path: Path

    def get_path(self) -> Path:
        # downloads artifact if necessary
        if not self.path.exists():
            download(self.url, path=self.path)

        return self.path

cached_conformers = Downloadable(
    url="https://chaiassets.com/chai1-inference-depencencies/conformers.apkl",
    path=repo_root.joinpath("downloads", "conformers.apkl"),
)'''

still it tries to download https://chaiassets.com/chai1-inference-depencencies/conformers.apkl

ChatGPT suggested

proxies = {
    'http': 'http://your_proxy:port',
    'https': 'https://your_proxy:port',
}
response = requests.get('https://chaiassets.com', proxies=proxies)

I will ask my system admin for proxy help

kimdn commented 1 week ago

Ok, when I use h100 at my institution's HPC, it keeps generating error (can't download https://chaiassets.com/chai1-inference-depencencies/conformers.apkl).

However, when I use a100 instead, no more conformers.apkl error as

(chai_lab) [kimd999@deception03 chai-lab]$ tail -f ps.log_a100_shared
downloading https://chaiassets.com/chai1-inference-depencencies/conformers.apkl
/people/kimd999/bin/Miniconda3-py3.12.4_Linux/lib/python3.12/site-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
  warnings.warn(
Loading checkpoint shards: 100%|██████████| 2/2 [00:00<00:00,  6.34it/s]
/people/kimd999/bin/Miniconda3-py3.12.4_Linux/lib/python3.12/site-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
  warnings.warn(
/people/kimd999/bin/Miniconda3-py3.12.4_Linux/lib/python3.12/site-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884
  warnings.warn(
/people/kimd999/bin/Miniconda3-py3.12.4_Linux/lib/python3.12/site-packages/torch/export/_unlift.py:58: UserWarning: Attempted to insert a get_attr Node with no underlying reference in the owning GraphModule! Call GraphModule.add_submodule to add the necessary submodule, GraphModule.add_parameter to add the necessary Parameter, or nn.Module.register_buffer to add the necessary buffer
  getattr_node = gm.graph.get_attr(lifted_node)
/people/kimd999/bin/Miniconda3-py3.12.4_Linux/lib/python3.12/site-packages/torch/fx/graph.py:1460: UserWarning: Node _lifted_tensor_constant0_1 target _lifted_tensor_constant0 _lifted_tensor_constant0 of  does not reference an nn.Module, nn.Parameter, or buffer, which is what 'get_attr' Nodes typically target
  warnings.warn(f'Node {node} target {node.target} {atom} of {seen_qualname} does '
/people/kimd999/bin/Miniconda3-py3.12.4_Linux/lib/python3.12/site-packages/torch/fx/graph.py:1460: UserWarning: Node _lifted_tensor_constant1_1 target _lifted_tensor_constant1 _lifted_tensor_constant1 of  does not reference an nn.Module, nn.Parameter, or buffer, which is what 'get_attr' Nodes typically target
  warnings.warn(f'Node {node} target {node.target} {atom} of {seen_qualname} does '
downloading https://chaiassets.com/chai1-inference-depencencies/models/512/feature_embedding.pt2
downloading https://chaiassets.com/chai1-inference-depencencies/models/512/token_input_embedder.pt2
downloading https://chaiassets.com/chai1-inference-depencencies/models/512/trunk.pt2
downloading https://chaiassets.com/chai1-inference-depencencies/models/512/diffusion_module.pt2
downloading https://chaiassets.com/chai1-inference-depencencies/models/512/confidence_head.pt2
Trunk recycles: 100%|██████████| 3/3 [00:10<00:00,  3.63s/it]
Diffusion steps: 100%|██████████| 199/199 [01:20<00:00,  2.46it/s]
Writing output to /tmp/outputs/pred.model_trunk_0_idx_0.pdb
Writing output to /tmp/outputs/pred.model_trunk_0_idx_1.pdb
Writing output to /tmp/outputs/pred.model_trunk_0_idx_2.pdb
Writing output to /tmp/outputs/pred.model_trunk_0_idx_3.pdb
Writing output to /tmp/outputs/pred.model_trunk_0_idx_4.pdb
Traceback (most recent call last):
  File "/qfs/people/kimd999/script/python/predict_structure/ligand_interaction/chai-lab/examples/predict_structure.py", line 39, in <module>
    scores = np.load(output_dir.joinpath("scores.model_idx_2.npz"))
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/people/kimd999/bin/Miniconda3-py3.12.4_Linux/lib/python3.12/site-packages/numpy/lib/npyio.py", line 427, in load
    fid = stack.enter_context(open(os_fspath(file), "rb"))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/outputs/scores.model_idx_2.npz'

I heard that my institution recently installed h100. Maybe that installed machine has a different proxy/url access system. Closing this ticket now.