MolecularAI / aizynthfinder

A tool for retrosynthetic planning
https://molecularai.github.io/aizynthfinder/
MIT License
559 stars 126 forks source link

Regarding with the error from performing download_public_data my_folder #157

Open kingljy0818 opened 1 month ago

kingljy0818 commented 1 month ago

Hello,

After upgrading AiZynthFinder to version 4.3, I encountered the following error while attempting to run download_public_data my_folder:

(aizynth-env) [root@localhost AiZynthFinder]# download_public_data my_folder Traceback (most recent call last): File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connection.py", line 198, in _new_conn sock = connection.create_connection( File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection raise err File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 793, in urlopen response = self._make_request( File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 491, in _make_request raise new_e File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 467, in _make_request self._validate_conn(conn) File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1099, in _validate_conn conn.connect() File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connection.py", line 616, in connect self.sock = sock = self._new_conn() File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connection.py", line 213, in _new_conn raise NewConnectionError( urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f6e51df13f0>: Failed to establish a new connection: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/requests/adapters.py", line 589, in send resp = conn.urlopen( File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 847, in urlopen retries = retries.increment( File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/util/retry.py", line 515, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='zenodo.org', port=443): Max retries exceeded with url: /record/7341155/files/uspto_keras_model.hdf5 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f6e51df13f0>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/Anaconda3/envs/aizynth-env/bin/download_public_data", line 8, in sys.exit(main()) File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/aizynthfinder/tools/download_public_data.py", line 89, in main _download_file(filespec["url"], os.path.join(path, filespec["filename"])) File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/aizynthfinder/tools/download_public_data.py", line 64, in _download_file with requests.get(url, stream=True) as response: File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/requests/api.py", line 73, in get return request("get", url, params=params, kwargs) File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, kwargs) File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, send_kwargs) File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, kwargs) File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/requests/adapters.py", line 622, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='zenodo.org', port=443): Max retries exceeded with url: /record/7341155/files/uspto_keras_model.hdf5 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f6e51df13f0>: Failed to establish a new connection: [Errno 111] Connection refused'))

I also tried using the previously downloaded 'my_folder', but when running the following code in the notebook:

from aizynthfinder.interfaces import AiZynthApp app = AiZynthApp("config.yml") it showed the following error:"


PolicyException Traceback (most recent call last) Cell In[1], line 2 1 from aizynthfinder.interfaces import AiZynthApp ----> 2 app = AiZynthApp("config.yml")

File /home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/aizynthfinder/interfaces/aizynthapp.py:58, in AiZynthApp.init(self, configfile, setup) 56 def init(self, configfile: str, setup: bool = True) -> None: 57 setup_logger(logging.INFO) ---> 58 self.finder = AiZynthFinder(configfile=configfile) 59 self._input: StrDict = dict() 60 self._output: StrDict = dict()

File /home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/aizynthfinder/aizynthfinder.py:71, in AiZynthFinder.init(self, configfile, configdict) 68 self._logger = logger() 70 if configfile: ---> 71 self.config = Configuration.from_file(configfile) 72 elif configdict: 73 self.config = Configuration.from_dict(configdict)

File /home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/aizynthfinder/context/config.py:141, in Configuration.from_file(cls, filename) 139 txt = txt.replace(item, os.environ[item[2:-1]]) 140 _config = yaml.load(txt, Loader=yaml.SafeLoader) --> 141 return Configuration.from_dict(_config)

File /home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/aizynthfinder/context/config.py:111, in Configuration.from_dict(cls, source) 108 config_obj._update_from_config(dict(source)) 110 config_obj.expansion_policy.load_from_config(expansion_config) --> 111 config_obj.filter_policy.load_from_config(filter_config) 112 config_obj.stock.load_from_config(**stock_config) 113 config_obj.scorers.create_default_scorers()

File /home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/aizynthfinder/context/policy/policies.py:223, in FilterPolicy.load_from_config(self, config) 221 if "type" in kwargs: 222 del kwargs["type"] --> 223 obj = cls(key, self._config, kwargs) 224 self.load(obj)

File /home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/aizynthfinder/context/policy/filter_strategies.py:112, in QuickKerasFilter.init(self, key, config, kwargs) 111 def init(self, key: str, config: Configuration, kwargs: Any) -> None: --> 112 super().init(key, config, **kwargs) 113 source = kwargs["model"] 114 # self.settings = self._config.filter_settings

File /home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/aizynthfinder/context/policy/filter_strategies.py:47, in FilterStrategy.init(self, key, config, kwargs) 45 def init(self, key: str, config: Configuration, kwargs: Any) -> None: 46 if any(name not in kwargs for name in self._required_kwargs): ---> 47 raise PolicyException( 48 f"A {self.class.name} class needs to be initiated " 49 f"with keyword arguments: {', '.join(self._required_kwargs)}" 50 ) 51 self._config = config 52 self._logger = logger()

PolicyException: A QuickKerasFilter class needs to be initiated with keyword arguments: model

How can I resolve this issue? I really need your help. Thank you!

Best regards,

Jiyuan

Lakshidaa commented 1 month ago

Hi @kingljy0818, Regarding the error you faced while performing download_public_data, we have now retired the downloading of public .hdf5 model files. However, the .onnx model files will still be downloaded. You can find the changes in our latest merge: https://github.com/MolecularAI/aizynthfinder/commit/c18cde13bc1b08bd327ee4d37021c514612021c8. This should resolve your error.

As per the PolicyException you encountered, can you please share more details on this? Specifically, the config.yml file that you used when you encountered this error.

kingljy0818 commented 1 month ago

Thank you very much for your response. I conducted a test and previously used version 3.7 of AiZynthFinder. At that time, running download_public_data my_folder did not result in the errors mentioned in the previous comment. It was possible to execute the following in a notebook:


from aizynthfinder.interfaces import AiZynthApp app = AiZynthApp("config.yml")

This would bring up the graphical interface. The specific contents of the config.yml used were:


policy: files: uspto:

However, upgrading to AiZynthFinder version 4.3 or doing a fresh installation of AiZynthFinder 4.3, and running download_public_idata my_folder, as well as executing the above in a notebook, both result in the errors previously mentioned. I really need your help to address these two issues in version 4.3. Thank you very much!

Lakshidaa commented 1 month ago

Hi @kingljy0818, It seems like you are using an old version of the config.yml, and this is probably because you are using version 3.7. I would suggest running download_public_data using the latest code on the master branch. The config.yml generated will reflect the new format and this will avoid the PolicyException you are currently encountering.

kingljy0818 commented 1 week ago

Hello, Lakshdaa,

Due to my server continuously testing REINVENT4.3 recently, my reply is delayed. Today, I downloaded the latest code of aizynthfinder, but encountered the following error when running download_public_data my_folder:

(aizynth-env) [root@localhost aizynthfinder-4.3.0]# download_public_data my_folder Traceback (most recent call last): File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connection.py", line 196, in _new_conn sock = connection.create_connection( File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/util/connection.py", line 85, in create_connection raise err File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/util/connection.py", line 73, in create_connection sock.connect(sa) ConnectionRefusedError: [Errno 111] Connection refused

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 789, in urlopen response = self._make_request( File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 490, in _make_request raise new_e File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 466, in _make_request self._validate_conn(conn) File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 1095, in _validate_conn conn.connect() File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connection.py", line 615, in connect self.sock = sock = self._new_conn() File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connection.py", line 211, in _new_conn raise NewConnectionError( urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x7f3196845300>: Failed to establish a new connection: [Errno 111] Connection refused


The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/requests/adapters.py", line 667, in send resp = conn.urlopen( File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/connectionpool.py", line 843, in urlopen retries = retries.increment( File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/urllib3/util/retry.py", line 519, in increment raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type] urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='zenodo.org', port=443): Max retries exceeded with url: /record/7341155/files/uspto_keras_model.hdf5 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f3196845300>: Failed to establish a new connection: [Errno 111] Connection refused'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/Anaconda3/envs/aizynth-env/bin/download_public_data", line 8, in sys.exit(main()) File "/home/AI_Synthesis/AiZynthFinder/aizynthfinder-4.3.0/aizynthfinder-4.3.0/aizynthfinder/tools/download_public_data.py", line 89, in main _download_file(filespec["url"], os.path.join(path, filespec["filename"])) File "/home/AI_Synthesis/AiZynthFinder/aizynthfinder-4.3.0/aizynthfinder-4.3.0/aizynthfinder/tools/download_public_data.py", line 64, in _download_file with requests.get(url, stream=True) as response: File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/requests/api.py", line 73, in get return request("get", url, params=params, kwargs) File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/requests/api.py", line 59, in request return session.request(method=method, url=url, kwargs) File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/requests/sessions.py", line 589, in request resp = self.send(prep, send_kwargs) File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/requests/sessions.py", line 703, in send r = adapter.send(request, kwargs) File "/home/Anaconda3/envs/aizynth-env/lib/python3.10/site-packages/requests/adapters.py", line 700, in send raise ConnectionError(e, request=request) requests.exceptions.ConnectionError: HTTPSConnectionPool(host='zenodo.org', port=443): Max retries exceeded with url: /record/7341155/files/uspto_keras_model.hdf5 (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x7f3196845300>: Failed to establish a new connection: [Errno 111] Connection refused'))


I also tested this in the Anaconda environment on Windows 11. When running download_public_data my_folder, the prompt was different from the one in the Linux environment:


(aizynth-env) PS D:\Lab\AI\AI_Synthesis\AiZynthFinder\aizynthfinder-4.3.0_Windows11_Download_Test\aizynthfinder-4.3.0\aizynthfinder-4.3.0> download_public_data my_folder uspto_model.hdf5: 0%| | 0.00/275M [00:00<?, ?B/s]Traceback (most recent call last): File "C:\Users\kingljy\anaconda3\envs\aizynth-env\lib\runpy.py", line 196, in _run_module_as_main return _run_code(code, main_globals, None, File "C:\Users\kingljy\anaconda3\envs\aizynth-env\lib\runpy.py", line 86, in _run_code exec(code, run_globals) File "C:\Users\kingljy\anaconda3\envs\aizynth-env\Scripts\download_public_data.exe__main__.py", line 7, in File "C:\Users\kingljy\anaconda3\envs\aizynth-env\lib\site-packages\aizynthfinder\tools\download_public_data.py", line 89, in main _download_file(filespec["url"], os.path.join(path, filespec["filename"])) File "C:\Users\kingljy\anaconda3\envs\aizynth-env\lib\site-packages\aizynthfinder\tools\download_public_data.py", line 70, in _download_file with open(filename, "wb") as fileobj: FileNotFoundError: [Errno 2] No such file or directory: 'my_folder\uspto_model.hdf5' uspto_model.hdf5: 0%|


I really need your help to resolve this issue, and I am also looking forward to upgrading from AiZynthFinder 3.7 to 4.3. Thank you very much.