Can't download dataset with downloader.py script

[IgnacioJPickering]

When trying to download the dataset using the downloader.py script I hit two issues:

• The recommended environment currently seems to install pydantic 2.0 by default, which is incompatible with qcportal. This is easily fixed by pinning the pydantic version to 1.10

• For some reason the connection is refused to the qcarchive server, no matter what I do. I'm not too familiar with the qcarchive api, any insights on how to debug this would be appreciated.

The traceback and output are:

Processing SPICE Solvated Amino Acids Single Points Dataset v1.1
Processing SPICE Dipeptides Single Points Dataset v1.2
Traceback (most recent call last):
  File "/spice-dataset/downloader/downloader.py", line 83, in <module>
    recs = ds.get_records(method=spec['method'], basis=spec['basis'], program=spec['program'], keywords=spec['keywords'])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/envs/spice/lib/python3.11/site-packages/qcportal/collections/dataset.py", line 1606, in get_records
    df = self._get_records(indexer, history, include=include, merge=merge, status=status)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/envs/spice/lib/python3.11/site-packages/qcportal/collections/dataset.py", line 1158, in _get_records
    records.extend(self.client.query_results(**query_set, status=status))
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/envs/spice/lib/python3.11/site-packages/qcportal/client.py", line 718, in query_results
    response = self._automodel_request("result", "get", payload, full_return=True)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/envs/spice/lib/python3.11/site-packages/qcportal/client.py", line 274, in _automodel_request
    r = self._request(rest, name, data=payload.serialize(self.encoding), timeout=timeout)
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/envs/spice/lib/python3.11/site-packages/qcportal/client.py", line 233, in _request
    raise ConnectionRefusedError(_connection_error_msg.format(self.address)) from None
ConnectionRefusedError: 

Could not connect to server https://api.qcarchive.molssi.org:443/, please check the address and try again.

[peastman]

No idea about the connection refused. Perhaps QCArchive is having temporary issues? I'd wait a little while and try again.

The recommended way of getting the dataset is to download it from Zenodo.

[IgnacioJPickering]

@peastman Thank you very much for the prompt response, the zenodo link works correctly. Looks like QCArchive is still having issues with my connection. Feel free to close the issue if you want.

[jchodera]

Copying in @bennybp, who might be able to help us update the SPICE download scripts for reproducibility.