Transformation products batch mode issues

[popenc]

Batch mode is throwing an error, "error requesting structure checker. Returning to inputs."

This can be recreated on ceam dev.

Logs: worker_cts_1 | [2022-09-09 18:31:31,618: WARNING/ForkPoolWorker-2] REQUEST POST: {'chemical': '', 'ph': '7.4', 'pchem_request': {'chemaxon': ['water_sol', 'water_sol_ph', 'ion_con', 'kow_no_ph', 'kow_wph']}, 'service': 'getTransProducts', 'workflow': 'gentrans', 'nodes': [{'chemical': 'C1=CC=CC=C1', 'orig_smiles': 'C1=CC=CC=C1', 'smiles': 'C1=CC=CC=C1', 'formula': 'C6H6', 'iupac': 'benzene', 'mass': 78.114, 'structureData': {'structure': '<?xml version="1.0" encoding="UTF-8"?>\n\n', 'format': 'mrv', 'contentUrl': 'http://qed-tomcat:8080/webservices/rest-v0/util/export/struc_798da45714b113fc488d5a3e17f2d8d0?format=mrv&contentName=structure', 'contentBaseUrl': 'http://qed-tomcat:8080/webservices/rest-v0/util/export/struc_798da45714b113fc488d5a3e17f2d8d0'}, 'exactMass': 78.0469501932, 'preferredName': 'benzene', 'cas': '27271-55-2, 68956-52-5, 26181-88-4, 174973-66-1, 54682-86-9, 71-43-2', 'has_carbon': True, 'dsstox': 'N/A', 'casrn': 'N/A', 'dtxsid': 'N/A'}, {'chemical': 'CC(=O)OC1=CC=CC=C1C(O)=O', 'orig_smiles': 'CC(=O)OC1=CC=CC=C1C(O)=O', 'smiles': 'CC(=O)OC1=CC=CC=C1C(O)=O', 'formula': 'C9H8O4', 'iupac': '2-(acetyloxy)benzoic acid', 'mass': 180.159, 'structureData': {'structure': '<?xml version="1.0" encoding="UTF-8"?>\n\n', 'format': 'mrv', 'contentUrl': 'http://qed-tomcat:8080/webservices/rest-v0/util/export/struc_fb858ddc480efb2f254e36cb0d034a32?format=mrv&contentName=structure', 'contentBaseUrl': 'http://qed-tomcat:8080/webservices/rest-v0/util/export/struc_fb858ddc480efb2f254e36cb0d034a32'}, 'exactMass': 180.042258738, 'preferredName': 'aspirin', 'cas': '50-78-2, 11126-35-5, 11126-37-7, 2349-94-2, 26914-13-6, 98201-60-6', 'has_carbon': True, 'dsstox': 'N/A', 'casrn': 'N/A', 'dtxsid': 'N/A'}, {'chemical': 'OC1=CC=CC=C1', 'orig_smiles': 'OC1=CC=CC=C1', 'smiles': 'OC1=CC=CC=C1', 'formula': 'C6H6O', 'iupac': 'phenol', 'mass': 94.113, 'structureData': {'structure': '<?xml version="1.0" encoding="UTF-8"?>\n\n', 'format': 'mrv', 'contentUrl': 'http://qed-tomcat:8080/webservices/rest-v0/util/export/struc_f502e1d049bc630b1e364f1608a5a776?format=mrv&contentName=structure', 'contentBaseUrl': 'http://qed-tomcat:8080/webservices/rest-v0/util/export/struc_f502e1d049bc630b1e364f1608a5a776'}, 'exactMass': 94.041864813, 'preferredName': 'phenol', 'cas': '1336-35-2, 108-95-2, 63496-48-0, 73607-76-8, 61788-41-8, 14534-23-7, 50356-25-7, 8002-07-1, 27073-41-2', 'has_carbon': True, 'dsstox': 'N/A', 'casrn': 'N/A', 'dtxsid': 'N/A'}, {'chemical': 'OC(=O)CC(O)(CC(O)=O)C(O)=O', 'orig_smiles': 'OC(=O)CC(O)(CC(O)=O)C(O)=O', 'smiles': 'OC(=O)CC(O)(CC(O)=O)C(O)=O', 'formula': 'C6H8O7', 'iupac': '2-hydroxypropane-1,2,3-tricarboxylic acid', 'mass': 192.123, 'structureData': {'structure': '<?xml version="1.0" encoding="UTF-8"?>\n\n', 'format': 'mrv', 'contentUrl': 'http://qed-tomcat:8080/webservices/rest-v0/util/export/struc_5030d675c68723c24bb817c4f8b816d8?format=mrv&contentName=structure', 'contentBaseUrl': 'http://qed-tomcat:8080/webservices/rest-v0/util/export/struc_5030d675c68723c24bb817c4f8b816d8'}, 'exactMass': 192.027002598, 'preferredName': 'citric acid', 'cas': '77-92-9, 12262-73-6, 245654-34-6, 43136-35-2', 'has_carbon': True, 'dsstox': 'N/A', 'casrn': 'N/A', 'dtxsid': 'N/A'}, {'chemical': '[O-]N+C1=CC=C(C=C1)N+=O', 'orig_smiles': '[O-]N+C1=CC=C(C=C1)N+=O', 'smiles': '[O-]N+C1=CC=C(C=C1)N+=O', 'formula': 'C6H4N2O4', 'iupac': '1,4-dinitrobenzene', 'mass': 168.108, 'structureData': {'structure': '<?xml version="1.0" encoding="UTF-8"?>\n\n', 'format': 'mrv', 'contentUrl': 'http://qed-tomcat:8080/webservices/rest-v0/util/export/struc_5df04bcb033228deee3a13136fbb9f5d?format=mrv&contentName=structure', 'contentBaseUrl': 'http://qed-tomcat:8080/webservices/rest-v0/util/export/struc_5df04bcb033228deee3a13136fbb9f5d'}, 'exactMass': 168.017106617, 'preferredName': 'P-dinitrobenzene', 'cas': '100-25-4', 'has_carbon': True, 'dsstox': 'N/A', 'casrn': 'N/A', 'dtxsid': 'N/A'}], 'run_type': 'batch', 'calc': 'chemaxon', 'gen_limit': 1, 'metabolizer_post': {'structure': '', 'generationLimit': 1, 'populationLimit': 0, 'likelyLimit': 0.1, 'excludeCondition': 'hasValenceError()', 'transformationLibraries': ['hydrolysis']}, 'sessionid': 'hZN8Fa18T_Z1CecRAAAL'} worker_cts_1 | [2022-09-09 18:31:32,243: INFO/ForkPoolWorker-2] metabolites: 1 worker_cts_1 | [2022-09-09 18:31:32,296: INFO/MainProcess] Received task: tasks.cts_task[3cad8369-07a3-4d91-aeef-2bde3edbcf11] expires:[2022-09-09 18:46:32.000293+00:00] worker_cts_1 | [2022-09-09 18:31:32,298: WARNING/ForkPoolWorker-1] REQUEST POST: {'chemical': '', 'is_node': True, 'service': 'getChemInfo', 'genKey': 'molecule 1', 'workflow': 'gentrans', 'sessionid': 'hZN8Fa18T_Z1CecRAAAL'} worker_cts_1 | [2022-09-09 18:31:32,327: WARNING/ForkPoolWorker-1] STRUCTURE CHECK INVALID: {'error': 'Chemical cannot be standardized', 'data': {'errorCode': 3, 'errorMessage': 'Cannot read molecule file.', 'rootCause': 'java.io.IOException', 'rootErrorMessage': 'Cannot read empty molecule file'}, 'valid': False} worker_cts_1 | [2022-09-09 18:31:32,332: INFO/ForkPoolWorker-1] Task tasks.cts_task[3cad8369-07a3-4d91-aeef-2bde3edbcf11] succeeded in 0.033908065408468246s: None worker_cts_1 | [2022-09-09 18:31:32,994: INFO/ForkPoolWorker-2] metabolites: 1 worker_cts_1 | [2022-09-09 18:31:32,994: INFO/ForkPoolWorker-2] metabolites: 2 worker_cts_1 | [2022-09-09 18:31:32,994: INFO/ForkPoolWorker-2] metabolites: 3 worker_cts_1 | [2022-09-09 18:31:33,053: INFO/MainProcess] Received task: tasks.cts_task[4aea5610-0dd1-46e0-9069-10f7bc02628a] expires:[2022-09-09 18:46:33.000050+00:00] worker_cts_1 | [2022-09-09 18:31:33,055: WARNING/ForkPoolWorker-1] REQUEST POST: {'chemical': '', 'is_node': True, 'service': 'getChemInfo', 'genKey': 'molecule 1', 'workflow': 'gentrans', 'sessionid': 'hZN8Fa18T_Z1CecRAAAL'} worker_cts_1 | [2022-09-09 18:31:33,076: INFO/MainProcess] Received task: tasks.cts_task[37548e17-fd57-4c34-a9fd-3769bd91d282] expires:[2022-09-09 18:46:33.000058+00:00] worker_cts_1 | [2022-09-09 18:31:33,079: INFO/MainProcess] Received task: tasks.cts_task[e7be32b2-0d12-4e84-b2ad-e6b8a4a36b19] expires:[2022-09-09 18:46:33.000059+00:00] worker_cts_1 | [2022-09-09 18:31:33,097: WARNING/ForkPoolWorker-1] STRUCTURE CHECK INVALID: {'error': 'Chemical cannot be standardized', 'data': {'errorCode': 3, 'errorMessage': 'Cannot read molecule file.', 'rootCause': 'java.io.IOException', 'rootErrorMessage': 'Cannot read empty molecule file'}, 'valid': False} worker_cts_1 | [2022-09-09 18:31:33,114: INFO/ForkPoolWorker-1] Task tasks.cts_task[4aea5610-0dd1-46e0-9069-10f7bc02628a] succeeded in 0.05967765301465988s: None worker_cts_1 | [2022-09-09 18:31:33,116: WARNING/ForkPoolWorker-1] REQUEST POST: {'chemical': 'OC(=O)c1ccccc1O', 'is_node': True, 'service': 'getChemInfo', 'genKey': 'molecule 1.1', 'workflow': 'gentrans', 'sessionid': 'hZN8Fa18T_Z1CecRAAAL'} worker_cts_1 | [2022-09-09 18:31:33,177: WARNING/ForkPoolWorker-1] VALID URL: http://qed-tomcat:8080/ctsws/rest/isvalidchemical worker_cts_1 | [2022-09-09 18:31:33,192: WARNING/ForkPoolWorker-1] VALID RESPONSE: <Response [200]> worker_cts_1 | [2022-09-09 18:31:33,192: WARNING/ForkPoolWorker-1] RESPONSE CONTENT: b'{"result":"true"}' worker_cts_1 | [2022-09-09 18:31:33,278: WARNING/ForkPoolWorker-1] Connection error for https://ccte-api-ccd.epa.gov/batchsearch/results.. No data from actorws.. worker_cts_1 | [2022-09-09 18:31:33,278: WARNING/ForkPoolWorker-1] Error getting CCTE data: 'NoneType' object is not subscriptable worker_cts_1 | [2022-09-09 18:31:33,279: WARNING/ForkPoolWorker-1] Using only Jchem WS results instead, setting dsstox value to N/A worker_cts_1 | [2022-09-09 18:31:33,317: WARNING/ForkPoolWorker-1] /usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py:842: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings worker_cts_1 | warnings.warn(( worker_cts_1 | worker_cts_1 | [2022-09-09 18:31:33,460: INFO/ForkPoolWorker-1] Task tasks.cts_task[37548e17-fd57-4c34-a9fd-3769bd91d282] succeeded in 0.34446507692337036s: None worker_cts_1 | [2022-09-09 18:31:33,462: WARNING/ForkPoolWorker-1] REQUEST POST: {'chemical': 'CC(O)=O', 'is_node': True, 'service': 'getChemInfo', 'genKey': 'molecule 1.2', 'workflow': 'gentrans', 'sessionid': 'hZN8Fa18T_Z1CecRAAAL'} worker_cts_1 | [2022-09-09 18:31:33,522: WARNING/ForkPoolWorker-1] VALID URL: http://qed-tomcat:8080/ctsws/rest/isvalidchemical worker_cts_1 | [2022-09-09 18:31:33,538: WARNING/ForkPoolWorker-1] VALID RESPONSE: <Response [200]> worker_cts_1 | [2022-09-09 18:31:33,538: WARNING/ForkPoolWorker-1] RESPONSE CONTENT: b'{"result":"true"}' worker_cts_1 | [2022-09-09 18:31:33,552: INFO/MainProcess] Received task: tasks.cts_task[10a433a6-b0ee-49b7-997f-d93bc0994647] expires:[2022-09-09 18:46:33.000549+00:00] worker_cts_1 | [2022-09-09 18:31:33,598: WARNING/ForkPoolWorker-1] Connection error for https://ccte-api-ccd.epa.gov/batchsearch/results.. No data from actorws.. worker_cts_1 | [2022-09-09 18:31:33,598: WARNING/ForkPoolWorker-1] Error getting CCTE data: 'NoneType' object is not subscriptable worker_cts_1 | [2022-09-09 18:31:33,599: WARNING/ForkPoolWorker-1] Using only Jchem WS results instead, setting dsstox value to N/A worker_cts_1 | [2022-09-09 18:31:33,684: INFO/ForkPoolWorker-2] metabolites: 1 worker_cts_1 | [2022-09-09 18:31:33,735: INFO/MainProcess] Received task: tasks.cts_task[9e31c2ce-aeea-4c30-8a55-0027758c18a5] expires:[2022-09-09 18:46:33.000732+00:00] worker_cts_1 | [2022-09-09 18:31:33,823: WARNING/ForkPoolWorker-1] /usr/local/lib/python3.9/site-packages/urllib3/connectionpool.py:842: InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings worker_cts_1 | warnings.warn(( worker_cts_1 | worker_cts_1 | [2022-09-09 18:31:33,987: INFO/ForkPoolWorker-1] Task tasks.cts_task[e7be32b2-0d12-4e84-b2ad-e6b8a4a36b19] succeeded in 0.5252015255391598s: None worker_cts_1 | [2022-09-09 18:31:33,993: WARNING/ForkPoolWorker-1] REQUEST POST: {'chemical': 'OC(=O)c1ccccc1O', 'ph': '7.4', 'pchem_request': {'chemaxon': ['water_sol', 'water_sol_ph', 'ion_con', 'kow_no_ph', 'kow_wph']}, 'run_type': 'batch', 'workflow': 'gentrans', 'genKey': 'molecule 1.1', 'mass': 138.122, 'request_post': {'service': None}, 'sessionid': 'hZN8Fa18T_Z1CecRAAAL', 'calc': 'chemaxon'} worker_cts_1 | [2022-09-09 18:31:34,039: INFO/MainProcess] Received task: tasks.cts_task[43c57667-9605-41a8-8eeb-67b63f9a67f4] expires:[2022-09-09 18:46:34.000028+00:00] worker_cts_1 | [2022-09-09 18:31:34,285: INFO/ForkPoolWorker-2] metabolites: 1 worker_cts_1 | [2022-09-09 18:31:34,341: INFO/MainProcess] Received task: tasks.cts_task[fe5b3730-6313-427f-bc41-bace78e06192] expires:[2022-09-09 18:46:34.000336+00:00] worker_cts_1 | [2022-09-09 18:31:34,880: INFO/ForkPoolWorker-2] metabolites: 1 worker_cts_1 | [2022-09-09 18:31:34,885: INFO/ForkPoolWorker-2] Task tasks.cts_task[786eb7cb-5251-49b2-ab7f-c48a6a68dd93] succeeded in 3.2674665674567223s: None worker_cts_1 | [2022-09-09 18:31:34,887: WARNING/ForkPoolWorker-2] REQUEST POST: {'chemical': '', 'is_node': True, 'service': 'getChemInfo', 'genKey': 'molecule 1', 'workflow': 'gentrans', 'sessionid': 'hZN8Fa18T_Z1CecRAAAL'} worker_cts_1 | [2022-09-09 18:31:34,909: WARNING/ForkPoolWorker-2] STRUCTURE CHECK INVALID: {'error': 'Chemical cannot be standardized', 'data': {'errorCode': 3, 'errorMessage': 'Cannot read molecule file.', 'rootCause': 'java.io.IOException', 'rootErrorMessage': 'Cannot read empty molecule file'}, 'valid': False} worker_cts_1 | [2022-09-09 18:31:34,914: INFO/ForkPoolWorker-2] Task tasks.cts_task[9e31c2ce-aeea-4c30-8a55-0027758c18a5] succeeded in 0.027499381452798843s: None worker_cts_1 | [2022-09-09 18:31:34,921: WARNING/ForkPoolWorker-2] REQUEST POST: {'chemical': 'CC(O)=O', 'ph': '7.4', 'pchem_request': {'chemaxon': ['water_sol', 'water_sol_ph', 'ion_con', 'kow_no_ph', 'kow_wph']}, 'run_type': 'batch', 'workflow': 'gentrans', 'genKey': 'molecule 1.2', 'mass': 60.052, 'request_post': {'service': None}, 'sessionid': 'hZN8Fa18T_Z1CecRAAAL', 'calc': 'chemaxon'} worker_cts_1 | [2022-09-09 18:31:34,945: INFO/MainProcess] Received task: tasks.cts_task[10a2e700-7e5a-43ce-8d11-7ca71e64bf81] expires:[2022-09-09 18:46:34.000942+00:00] worker_cts_1 | [2022-09-09 18:31:35,086: WARNING/ForkPoolWorker-2] cts_celery calculator_chemaxon -- jchem server response: <Response [500]> worker_cts_1 | b'{"errorCode":0,"errorMessage":"4","rootCause":"java.lang.ArrayIndexOutOfBoundsException"}' worker_cts_1 | [2022-09-09 18:31:35,086: INFO/ForkPoolWorker-2] Max retries: 3, Retries left: 1 worker_cts_1 | [2022-09-09 18:31:35,292: WARNING/ForkPoolWorker-1] cts_celery calculator_chemaxon -- jchem server response: <Response [500]> worker_cts_1 | b'{"errorCode":0,"errorMessage":"Array index out of range: 9","rootCause":"java.lang.ArrayIndexOutOfBoundsException"}' worker_cts_1 | [2022-09-09 18:31:35,292: INFO/ForkPoolWorker-1] Max retries: 3, Retries left: 1 worker_cts_1 | [2022-09-09 18:31:35,481: WARNING/ForkPoolWorker-2] cts_celery calculator_chemaxon -- jchem server response: <Response [500]> worker_cts_1 | b'{"errorCode":0,"errorMessage":"java.lang.ArrayIndexOutOfBoundsException","rootCause":"java.lang.ArrayIndexOutOfBoundsException"}' worker_cts_1 | [2022-09-09 18:31:35,482: INFO/ForkPoolWorker-2] Max retries: 3, Retries left: 2 worker_cts_1 | [2022-09-09 18:31:35,486: WARNING/ForkPoolWorker-1] cts_celery calculator_chemaxon -- jchem server response: <Response [500]> worker_cts_1 | b'{"errorCode":0,"errorMessage":"1","rootCause":"java.lang.ArrayIndexOutOfBoundsException"}' worker_cts_1 | [2022-09-09 18:31:35,486: INFO/ForkPoolWorker-1] Max retries: 3, Retries left: 2 worker_cts_1 | [2022-09-09 18:31:35,519: WARNING/ForkPoolWorker-1] cts_celery calculator_chemaxon -- jchem server response: <Response [400]> worker_cts_1 | b'{"errorCode":3,"errorMessage":"Exception in fragment search.","rootCause":"java.lang.NullPointerException"}' worker_cts_1 | [2022-09-09 18:31:35,519: INFO/ForkPoolWorker-1] Max retries: 3, Retries left: 3 worker_cts_1 | [2022-09-09 18:31:35,520: WARNING/ForkPoolWorker-1] Exception occurred getting chemaxon data: 'NoneType' object is not subscriptable worker_cts_1 | [2022-09-09 18:31:35,527: WARNING/ForkPoolWorker-2] cts_celery calculator_chemaxon -- jchem server response: <Response [400]> worker_cts_1 | b'{"errorCode":3,"errorMessage":"Exception in fragment search.","rootCause":"java.lang.IllegalArgumentException","rootErrorMessage":"layer: SEARCH_INTERNAL is already defined"}' worker_cts_1 | [2022-09-09 18:31:35,527: INFO/ForkPoolWorker-2] Max retries: 3, Retries left: 3 worker_cts_1 | [2022-09-09 18:31:35,527: WARNING/ForkPoolWorker-2] Exception occurred getting chemaxon data: 'NoneType' object is not subscriptable worker_cts_1 | [2022-09-09 18:31:35,592: WARNING/ForkPoolWorker-2] cts_celery calculator_chemaxon -- jchem server response: <Response [400]> worker_cts_1 | b'{"errorCode":3,"errorMessage":"Error during query initialization: A higher layer than TRANSFORMED is already defined: SEARCH_INTERNAL","rootCause":"java.lang.IllegalArgumentException","rootErrorMessage":"A higher layer than TRANSFORMED is already defined: SEARCH_INTERNAL"}' worker_cts_1 | [2022-09-09 18:31:35,592: INFO/ForkPoolWorker-2] Max retries: 3, Retries left: 1 worker_cts_1 | [2022-09-09 18:31:35,619: WARNING/ForkPoolWorker-2] cts_celery calculator_chemaxon -- jchem server response: <Response [400]> worker_cts_1 | b'{"errorCode":3,"errorMessage":"Error during query initialization: null","rootCause":"java.util.NoSuchElementException"}' worker_cts_1 | [2022-09-09 18:31:35,619: INFO/ForkPoolWorker-2] Max retries: 3, Retries left: 2 worker_cts_1 | [2022-09-09 18:31:35,633: WARNING/ForkPoolWorker-1] cts_celery calculator_chemaxon -- jchem server response: <Response [400]> worker_cts_1 | b'{"errorCode":3,"errorMessage":"Exception in fragment search.","rootCause":"java.lang.IllegalArgumentException","rootErrorMessage":"layer: SEARCH_INTERNAL is already defined"}' worker_cts_1 | [2022-09-09 18:31:35,633: INFO/ForkPoolWorker-1] Max retries: 3, Retries left: 1 worker_cts_1 | [2022-09-09 18:31:35,661: WARNING/ForkPoolWorker-2] cts_celery calculator_chemaxon -- jchem server response: <Response [500]> worker_cts_1 | b'{"errorCode":0,"errorMessage":"Index: 8, Size: 4","rootCause":"java.lang.IndexOutOfBoundsException"}' worker_cts_1 | [2022-09-09 18:31:35,661: INFO/ForkPoolWorker-2] Max retries: 3, Retries left: 3 worker_cts_1 | [2022-09-09 18:31:35,661: WARNING/ForkPoolWorker-2] Exception occurred getting chemaxon data: 'NoneType' object is not subscriptable worker_cts_1 | [2022-09-09 18:31:36,606: INFO/ForkPoolWorker-2] Task tasks.cts_task[43c57667-9605-41a8-8eeb-67b63f9a67f4] succeeded in 1.6853183023631573s: None worker_cts_1 | [2022-09-09 18:31:36,608: WARNING/ForkPoolWorker-2] REQUEST POST: {'chemical': '', 'is_node': True, 'service': 'getChemInfo', 'genKey': 'molecule 1', 'workflow': 'gentrans', 'sessionid': 'hZN8Fa18T_Z1CecRAAAL'} worker_cts_1 | [2022-09-09 18:31:36,640: WARNING/ForkPoolWorker-2] STRUCTURE CHECK INVALID: {'error': 'Chemical cannot be standardized', 'data': {'errorCode': 3, 'errorMessage': 'Cannot read molecule file.', 'rootCause': 'java.io.IOException', 'rootErrorMessage': 'Cannot read empty molecule file'}, 'valid': False} worker_cts_1 | [2022-09-09 18:31:36,645: INFO/ForkPoolWorker-2] Task tasks.cts_task[fe5b3730-6313-427f-bc41-bace78e06192] succeeded in 0.03725946694612503s: None worker_cts_1 | [2022-09-09 18:31:36,648: WARNING/ForkPoolWorker-2] REQUEST POST: {'chemical': '', 'is_node': True, 'service': 'getChemInfo', 'genKey': 'molecule 1', 'workflow': 'gentrans', 'sessionid': 'hZN8Fa18T_Z1CecRAAAL'} worker_cts_1 | [2022-09-09 18:31:36,676: WARNING/ForkPoolWorker-2] STRUCTURE CHECK INVALID: {'error': 'Chemical cannot be standardized', 'data': {'errorCode': 3, 'errorMessage': 'Cannot read molecule file.', 'rootCause': 'java.io.IOException', 'rootErrorMessage': 'Cannot read empty molecule file'}, 'valid': False} worker_cts_1 | [2022-09-09 18:31:36,681: INFO/ForkPoolWorker-2] Task tasks.cts_task[10a2e700-7e5a-43ce-8d11-7ca71e64bf81] succeeded in 0.033382631838321686s: None worker_cts_1 | [2022-09-09 18:31:37,395: INFO/ForkPoolWorker-1] Task tasks.cts_task[10a433a6-b0ee-49b7-997f-d93bc0994647] succeeded in 3.4013859070837498s: None worker_cts_1 | [2022-09-09 18:31:38,818: INFO/MainProcess] Tasks flagged as revoked: 786eb7cb-5251-49b2-ab7f-c48a6a68dd93 worker_cts_1 | [2022-09-09 18:31:38,819: INFO/MainProcess] Tasks flagged as revoked: 3cad8369-07a3-4d91-aeef-2bde3edbcf11 worker_cts_1 | [2022-09-09 18:31:38,821: INFO/MainProcess] Tasks flagged as revoked: 4aea5610-0dd1-46e0-9069-10f7bc02628a worker_cts_1 | [2022-09-09 18:31:38,821: INFO/MainProcess] Tasks flagged as revoked: 37548e17-fd57-4c34-a9fd-3769bd91d282 worker_cts_1 | [2022-09-09 18:31:38,824: INFO/MainProcess] Tasks flagged as revoked: e7be32b2-0d12-4e84-b2ad-e6b8a4a36b19 worker_cts_1 | [2022-09-09 18:31:38,827: INFO/MainProcess] Tasks flagged as revoked: 10a433a6-b0ee-49b7-997f-d93bc0994647 worker_cts_1 | [2022-09-09 18:31:38,829: INFO/MainProcess] Tasks flagged as revoked: 9e31c2ce-aeea-4c30-8a55-0027758c18a5 worker_cts_1 | [2022-09-09 18:31:38,829: INFO/MainProcess] Tasks flagged as revoked: 43c57667-9605-41a8-8eeb-67b63f9a67f4 worker_cts_1 | [2022-09-09 18:31:38,831: INFO/MainProcess] Tasks flagged as revoked: fe5b3730-6313-427f-bc41-bace78e06192 worker_cts_1 | [2022-09-09 18:31:38,833: INFO/MainProcess] Tasks flagged as revoked: 10a2e700-7e5a-43ce-8d11-7ca71e64bf81

[popenc]

Looks like some missing 'chemical' values are occurring.

[popenc]

Resolves transformation products, but getting some issues with water solubility for ChemAxon:

qed-worker_cts-1 | b'{"errorCode":3,"errorMessage":"Error during query initialization: null","rootCause":"java.util.NoSuchElementException"}'

qed-worker_cts-1 | [2022-09-12 18:47:42,759: INFO/ForkPoolWorker-1] Max retries: 3, Retries left: 1

qed-worker_cts-1 | [2022-09-12 18:47:42,816: WARNING/ForkPoolWorker-2] cts_celery calculator_chemaxon -- jchem server response: <Response [400]>

qed-worker_cts-1 | b'{"errorCode":3,"errorMessage":"Exception in fragment search.","rootCause":"java.lang.IllegalArgumentException","rootErrorMessage":"layer: SEARCH_INTERNAL is already defined"}'

qed-worker_cts-1 | [2022-09-12 18:47:42,816: INFO/ForkPoolWorker-2] Max retries: 3, Retries left: 2

qed-worker_cts-1 | [2022-09-12 18:47:42,883: WARNING/ForkPoolWorker-1] cts_celery calculator_chemaxon -- jchem server response: <Response [400]>

qed-worker_cts-1 | b'{"errorCode":3,"errorMessage":"Exception in fragment search.","rootCause":"java.lang.ArrayIndexOutOfBoundsException","rootErrorMessage":"-1"}'

qed-worker_cts-1 | [2022-09-12 18:47:42,883: INFO/ForkPoolWorker-1] Max retries: 3, Retries left: 2

qed-worker_cts-1 | [2022-09-12 18:47:42,956: WARNING/ForkPoolWorker-1] cts_celery calculator_chemaxon -- jchem server response: <Response [400]>

qed-worker_cts-1 | b'{"errorCode":3,"errorMessage":"Error during query initialization: null","rootCause":"java.lang.NullPointerException"}'

qed-worker_cts-1 | [2022-09-12 18:47:42,957: INFO/ForkPoolWorker-1] Max retries: 3, Retries left: 3

qed-worker_cts-1 | [2022-09-12 18:47:42,957: WARNING/ForkPoolWorker-1] Exception occurred getting chemaxon data: 'NoneType' object is not subscriptable

qed-worker_cts-1 | [2022-09-12 18:47:42,957: WARNING/ForkPoolWorker-2] cts_celery calculator_chemaxon -- jchem server response: <Response [400]>

qed-worker_cts-1 | b'{"errorCode":3,"errorMessage":"Error during query initialization: null","rootCause":"java.lang.NullPointerException"}'

qed-worker_cts-1 | [2022-09-12 18:47:42,957: INFO/ForkPoolWorker-2] Max retries: 3, Retries left: 3

qed-worker_cts-1 | [2022-09-12 18:47:42,957: WARNING/ForkPoolWorker-2] Exception occurred getting chemaxon data: 'NoneType' object is not subscriptable

qed-worker_cts-1 | [2022-09-12 18:47:42,970: WARNING/ForkPoolWorker-1] cts_celery calculator_chemaxon -- jchem server response: <Response [400]>

qed-worker_cts-1 | b'{"errorCode":3,"errorMessage":"Error during query initialization: layer: BASEMOLSEARCH_CORE_INPUT is already defined","rootCause":"java.lang.IllegalArgumentException","rootErrorMessage":"layer: BASEMOLSEARCH_CORE_INPUT is already defined"}'

qed-worker_cts-1 | [2022-09-12 18:47:42,970: INFO/ForkPoolWorker-1] Max retries: 3, Retries left: 1

qed-worker_cts-1 | [2022-09-12 18:47:42,970: WARNING/ForkPoolWorker-2] cts_celery calculator_chemaxon -- jchem server response: <Response [400]>

qed-worker_cts-1 | b'{"errorCode":3,"errorMessage":"Exception in fragment search.","rootCause":"java.lang.IllegalArgumentException","rootErrorMessage":"Unknown MolStack MoleculeLayer: BASEMOLSEARCH_CORE_INPUT"}'

qed-worker_cts-1 | [2022-09-12 18:47:42,970: INFO/ForkPoolWorker-2] Max retries: 3, Retries left: 1

qed-worker_cts-1 | [2022-09-12 18:47:42,981: WARNING/ForkPoolWorker-1] cts_celery calculator_chemaxon -- jchem server response: <Response [400]>

qed-worker_cts-1 | b'{"errorCode":3,"errorMessage":"Error during query initialization: layer: TRANSFORMED is already defined","rootCause":"java.lang.IllegalArgumentException","rootErrorMessage":"layer: TRANSFORMED is already defined"}'

qed-worker_cts-1 | [2022-09-12 18:47:42,981: INFO/ForkPoolWorker-1] Max retries: 3, Retries left: 2

qed-worker_cts-1 | [2022-09-12 18:47:42,983: WARNING/ForkPoolWorker-2] cts_celery calculator_chemaxon -- jchem server response: <Response [400]>

[popenc]

After revisiting this, it looks like EPA production has no issues processing the batch requests, but on ceam dev the request never completes. Looking further into this, it seems that EPI Suite in the ceam dev environment is not returning the expected number of properties back. Looks like prod returns two values for "Organic Carbon Partition Coefficient", e.g., 1.00 MCI and -1.16 KOW, but ceam dev is returning one value: 10.00 (without any units).

So it looks like the new EPI Suite updates are returning one value for that property instead of two.

[popenc]

Batch transformation products working again after epi suite fix. Tested with the sample batch file.