Open cristianvasquez opened 5 months ago
The transformation happens by batches.
There are be cases where some notices from a batch fail, but this is reported in Metabase as a success. This might be reported as a partial success.
Extract of the logs: (60mb)
NoticeEventMessage :: {'message': 'This notice 128456-2024 is not eligible for transformation. Notice info: form_number=[], eform_subtype=[29], xsd_version=[None]. Check mapping suites!', 'created_at': datetime.datetime(2024, 6, 17, 13, 7, 24, 130267), 'year': 2024, 'month': 6, 'day': 17, 'severity_level': 40, 'caller_name': 'log_notice_error', 'started_at': datetime.datetime(2024, 6, 17, 13, 7, 24, 130236), 'ended_at': datetime.datetime(2024, 6, 17, 13, 7, 24, 130237), 'duration': None, 'metadata': None, 'kwargs': None, 'notice_id': '128456-2024', 'domain_action': 'notice_transformation_pipeline', 'notice_form_number': '', 'notice_eforms_subtype': '29', 'notice_status': 'INELIGIBLE_FOR_TRANSFORMATION'}\n[2024-06-17T13:07:24.261+0000] {log.py:232} WARNING - [2024-06-17 13:07:24] - ConsoleHandler:ROOT - INFO - NoticeEventMessage :: {'message': None, 'created_at': datetime.datetime(2024, 6, 17, 13, 7, 23, 990381), 'year': 2024, 'month': 6, 'day': 17, 'severity_level': 20, 'caller_name': 'execute', 'started_at': datetime.datetime(2024, 6, 17, 13, 7, 23, 990403), 'ended_at': datetime.datetime(2024, 6, 17, 13, 7, 24, 256300), 'duration': 0.265897, 'metadata': None, 'kwargs': None, 'notice_id': '128456-2024', 'domain_action': 'notice_transformation_pipeline', 'notice_form_number': '', 'notice_eforms_subtype': '29', 'notice_status': 'INELIGIBLE_FOR_TRANSFORMATION'}\n[2024-06-17T13:07:24.585+0000] {log.py:232} WARNING - [2024-06-17 13:07:24] - ConsoleHandler:ROOT - ERROR - NoticeEventMessage :: {'message': 'This notice 129588-2024 is not eligible for transformation. Notice info: form_number=[], eform_subtype=[29], xsd_version=[None]. Check mapping suites!', 'created_at': datetime.datetime(2024, 6, 17, 13, 7, 24, 582226), 'year': 2024, 'month': 6, 'day': 17, 'severity_level': 40, 'caller_name': 'log_notice_error', 'started_at': datetime.datetime(2024, 6, 17, 13, 7, 24, 582185), 'ended_at': datetime.datetime(2024, 6, 17, 13, 7, 24, 582186), 'duration': None, 'metadata': None, 'kwargs': None, 'notice_id': '129588-2024', 'domain_action': 'notice_transformation_pipeline', 'notice_form_number': '', 'notice_eforms_subtype': '29', 'notice_status': 'INELIGIBLE_FOR_TRANSFORMATION'}\n[2024-06-17T13:07:24.818+0000] {log.py:232} WARNING - [2024-06-17 13:07:24] - ConsoleHandler:ROOT - INFO - NoticeEventMessage :: {'message': None, 'created_at': datetime.datetime(2024, 6, 17, 13, 7, 24, 441550), 'year': 2024, 'month': 6, 'day': 17, 'severity_level': 20, 'caller_name': 'execute', 'started_at': datetime.datetime(2024, 6, 17, 13, 7, 24, 441591), 'ended_at': datetime.datetime(2024, 6, 17, 13, 7, 24, 788149), 'duration': 0.346558, 'metadata': None, 'kwargs': None, 'notice_id': '129588-2024', 'domain_action': 'notice_transformation_pipeline', 'notice_form_number': '', 'notice_eforms_subtype': '29', 'notice_status': 'INELIGIBLE_FOR_TRANSFORMATION'}\n[2024-06-17T13:07:25.175+0000] {log.py:232} WARNING - [2024-06-17 13:07:25] - ConsoleHandler:ROOT - ERROR - NoticeEventMessage :: {'message': 'This notice 128544-2024 is not eligible for transformation. Notice info: form_number=[], eform_subtype=[38], xsd_version=[None]. Check mapping suites!', 'created_at': datetime.datetime(2024, 6, 17, 13, 7, 25, 135075), 'year': 2024, 'month': 6, 'day': 17, 'severity_level': 40, 'caller_name': 'log_notice_error', 'started_at': datetime.datetime(2024, 6, 17, 13, 7, 25, 135039), 'ended_at': datetime.datetime(2024, 6, 17, 13, 7, 25, 135041), 'duration': None, 'metadata': None, 'kwargs': None, 'notice_id': '128544-2024', 'domain_action': 'notice_transformation_pipeline', 'notice_form_number': '', 'notice_eforms_subtype': '38', 'notice_status': 'INELIGIBLE_FOR_TRANSFORMATION'}\n[2024-06-17T13:07:25.497+0000] {log.py:232} WARNING - [2024-06-17 13:07:25] - ConsoleHandler:ROOT - INFO - NoticeEventMessage :: {'message': None, 'created_at': datetime.datetime(2024, 6, 17, 13, 7, 24, 909113), 'year': 2024, 'month': 6, 'day': 17, 'severity_level': 20, 'caller_name': 'execute', 'started_at': datetime.datetime(2024, 6, 17, 13, 7, 24, 909133), 'ended_at': datetime.datetime(2024, 6, 17, 13, 7, 25, 493854), 'duration': 0.584721, 'metadata': None, 'kwargs': None, 'notice_id': '128544-2024', 'domain_action': 'notice_transformation_pipeline', 'notice_form_number': '', 'notice_eforms_subtype': '38', 'notice_status': 'INELIGIBLE_FOR_TRANSFORMATION'}\n[2024-06-17T13:25:53.402+0000] {log.py:232} WARNING - [2024-06-17 13:25:53] - ConsoleHandler:ROOT - ERROR - NoticeEventMessage :: {'message': 'net.sf.saxon.s9api.SaxonApiException: A sequence of more than one item is not allowed as the first argument of fn:path() (<cac:ProcurementProjectLot>, <cac:ProcurementProjectLot>) \\n\\tat net.sf.saxon.s9api.XPathSelector.evaluate(XPathSelector.java:233)\\n\\tat net.sf.saxon.s9api.XPathCompiler.evaluate(XPathCompiler.java:610)\\n\\tat be.ugent.rml.records.XMLRecord.get(XMLRecord.java:37)\\n\\tat be.ugent.rml.extractor.ReferenceExtractor.extract(ReferenceExtractor.java:31)\\n\\tat
The transformation happens by batches.
There are be cases where some notices from a batch fail, but this is reported in Metabase as a success. This might be reported as a partial success.
Extract of the logs: (60mb)