search

Call-for-Code-for-Racial-Justice / Legit-Info

"Legit Info" app helps residents understand the impact of legislation and policies based on their country, state, county, city, district location
Apache License 2.0
66 stars 74 forks source link

Error during b_extract_files cron job. #72

Closed tonypearson closed 3 years ago

tonypearson commented 3 years ago

Error during b_extract_files processing:

05-26 20:48:47 tasks:b_extract_files [INFO] 63:task started: b_extract_files [nltk_data] Downloading package punkt to /home/tpearson/nltk_data... [nltk_data] Package punkt is already up-to-date! 05-26 20:48:50 log_time:time_now [INFO] Starting extract_files at May-26 01:48PM MST 05-26 20:48:50 extract_files:handle [INFO] 194:Processing: United States (US) 05-26 20:49:31 extract_files:write_file [INFO] 478:Writing: US-HB0003-1823-Y2021.txt 05-26 20:49:37 extract_files:write_file [INFO] 478:Writing: US-HB0006-1823-Y2021.txt 05-26 20:49:39 extract_files:write_file [INFO] 478:Writing: US-HB0008-1823-Y2021.txt 05-26 20:49:49 extract_files:write_file [INFO] 478:Writing: US-HB0019-1823-Y2021.txt 05-26 20:49:54 extract_files:write_file [INFO] 478:Writing: US-HB0033-1823-Y2021.txt 05-26 20:50:10 extract_files:write_file [INFO] 478:Writing: US-HB0131-1823-Y2021.txt 05-26 20:50:18 extract_files:write_file [INFO] 478:Writing: US-HB0159-1823-Y2021.txt 05-26 20:50:36 extract_files:write_file [INFO] 478:Writing: US-HB0287-1823-Y2021.txt 05-26 20:50:44 extract_files:write_file [INFO] 478:Writing: US-HB0331-1823-Y2021.txt 05-26 20:50:54 extract_files:write_file [INFO] 478:Writing: US-HB0395-1823-Y2021.txt 05-26 20:50:54 extract_files:handle [INFO] 194:Processing: Arizona (AZ) 05-26 21:01:21 extract_files:handle [INFO] 194:Processing: Ohio (OH) 05-26 21:01:23 extract_files:process_location [ERROR] 238: Process Error. HTTPSConnectionPool(host='search-prod.lis.state.oh.us', port=443): Max retries exceeded with url: /solarapi/v1/general_assembly_134/bills/hb1/IN/00?format=pdf (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)'),)) Traceback (most recent call last): File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/urllib3/connectionpool.py", line 706, in urlopen chunked=chunked, File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/urllib3/connectionpool.py", line 382, in _make_request self._validate_conn(conn) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn conn.connect() File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/urllib3/connection.py", line 421, in connect tls_in_tls=tls_in_tls, File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15m/lib/python3.6/site-packages/urllib3/util/ssl.py", line 429, in ssl_wrap_socket sock, context, tls_in_tls, server_hostname=server_hostname File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15m/lib/python3.6/site-packages/urllib3/util/ssl.py", line 472, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock, server_hostname=server_hostname) File "/usr/lib64/python3.6/ssl.py", line 365, in wrap_socket _context=self, _session=session) File "/usr/lib64/python3.6/ssl.py", line 776, in init self.do_handshake() File "/usr/lib64/python3.6/ssl.py", line 1036, in do_handshake self._sslobj.do_handshake() File "/usr/lib64/python3.6/ssl.py", line 648, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/urllib3/connectionpool.py", line 756, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/urllib3/util/retry.py", line 574, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='search-prod.lis.state.oh.us', port=443): Max retries exceeded with url: /solarapi/v1/general_assembly_134/bills/hb1/IN/00?format=pdf (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)'),))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 186, in process_location self.process_json(json_name) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 256, in process_json self.process_zip(msg_bytes) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 282, in process_zip processed = self.process_source(json_data) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 326, in process_source processed = self.skip_if_exists(text_name, detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 383, in skip_if_exists processed = self.process_bill(detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 414, in process_bill bindata = self.fetch_state_link(detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 441, in fetch_state_link response = bill_bundle.make_request(baseurl, params) File "/home/tpearson/Develop/Legit-Info/cfc_app/data_bundle.py", line 63, in make_request response = requests.get(url, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/requests/api.py", line 76, in get return request('get', url, params=params, kwargs) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, kwargs) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/requests/sessions.py", line 542, in request resp = self.send(prep, send_kwargs) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/requests/sessions.py", line 655, in send r = adapter.send(request, kwargs) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/requests/adapters.py", line 514, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='search-prod.lis.state.oh.us', port=443): Max retries exceeded with url: /solarapi/v1/general_assembly_134/bills/hb1/IN/00?format=pdf (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)'),)) 21:01:23 [Q] ERROR Failed [cat-uniform-louisiana-floor] - 238: Process Error. HTTPSConnectionPool(host='search-prod.lis.state.oh.us', port=443): Max retries exceeded with url: /solarapi/v1/general_assembly_134/bills/hb1/IN/00?format=pdf (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)'),)) : Traceback (most recent call last): File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/urllib3/connectionpool.py", line 706, in urlopen chunked=chunked, File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/urllib3/connectionpool.py", line 382, in _make_request self._validate_conn(conn) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/urllib3/connectionpool.py", line 1010, in _validate_conn conn.connect() File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/urllib3/connection.py", line 421, in connect tls_in_tls=tls_in_tls, File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15m/lib/python3.6/site-packages/urllib3/util/ssl.py", line 429, in ssl_wrap_socket sock, context, tls_in_tls, server_hostname=server_hostname File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15m/lib/python3.6/site-packages/urllib3/util/ssl.py", line 472, in _ssl_wrap_socket_impl return ssl_context.wrap_socket(sock, server_hostname=server_hostname) File "/usr/lib64/python3.6/ssl.py", line 365, in wrap_socket _context=self, _session=session) File "/usr/lib64/python3.6/ssl.py", line 776, in init self.do_handshake() File "/usr/lib64/python3.6/ssl.py", line 1036, in do_handshake self._sslobj.do_handshake() File "/usr/lib64/python3.6/ssl.py", line 648, in do_handshake self._sslobj.do_handshake() ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/requests/adapters.py", line 449, in send timeout=timeout File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/urllib3/connectionpool.py", line 756, in urlopen method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2] File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/urllib3/util/retry.py", line 574, in increment raise MaxRetryError(_pool, url, error or ResponseError(cause)) urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='search-prod.lis.state.oh.us', port=443): Max retries exceeded with url: /solarapi/v1/general_assembly_134/bills/hb1/IN/00?format=pdf (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)'),))

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 186, in process_location self.process_json(json_name) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 256, in process_json self.process_zip(msg_bytes) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 282, in process_zip processed = self.process_source(json_data) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 326, in process_source processed = self.skip_if_exists(text_name, detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 383, in skip_if_exists processed = self.process_bill(detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 414, in process_bill bindata = self.fetch_state_link(detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 441, in fetch_state_link response = bill_bundle.make_request(baseurl, params) File "/home/tpearson/Develop/Legit-Info/cfc_app/data_bundle.py", line 63, in make_request response = requests.get(url, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/requests/api.py", line 76, in get return request('get', url, params=params, kwargs) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/requests/api.py", line 61, in request return session.request(method=method, url=url, kwargs) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/requests/sessions.py", line 542, in request resp = self.send(prep, send_kwargs) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/requests/sessions.py", line 655, in send r = adapter.send(request, kwargs) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/requests/adapters.py", line 514, in send raise SSLError(e, request=request) requests.exceptions.SSLError: HTTPSConnectionPool(host='search-prod.lis.state.oh.us', port=443): Max retries exceeded with url: /solarapi/v1/general_assembly_134/bills/hb1/IN/00?format=pdf (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)'),))

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django_q/cluster.py", line 436, in worker res = f(*task["args"], task["kwargs"]) File "/home/tpearson/Develop/Legit-Info/cfc_app/tasks.py", line 69, in b_extract_files call_command(cmd, '--api', '--skip') File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/core/management/init.py", line 168, in call_command return command.execute(*args, *defaults) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/core/management/base.py", line 371, in execute output = self.handle(args, options) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 144, in handle self.process_location(state) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 190, in process_location raise ExtractTextError(err_msg) from exc cfc_app.management.commands.extract_files.ExtractTextError: 238: Process Error. HTTPSConnectionPool(host='search-prod.lis.state.oh.us', port=443): Max retries exceeded with url: /solarapi/v1/general_assembly_134/bills/hb1/IN/00?format=pdf (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)'),))

tonypearson commented 3 years ago

Problem appears to be related to SSL: 05-26 21:01:22 extract_files:process_location [DEBUG] 234:Session_id=1815 JSON=OH-Dataset-1815.json 05-26 21:01:22 extract_files:process_json [DEBUG] 209:Checking JSON: OH-Dataset-1815.json 05-26 21:01:22 extract_files:process_json [DEBUG] Hashcode for OH-Dataset-1815.zip saved. 05-26 21:01:23 extract_files:process_zip [DEBUG] 315:PATH name: OH/2021-2022_134th_General_Assembly/bill/HB1.json 05-26 21:01:23 extract_files:process_zip [DEBUG] 320:JD: {"bill":{"bill_id":1437460,"change_hash":"c28191e345e6e55cb0241dc926bb1286" 05-26 21:01:23 extract_files:process_source [DEBUG] 289:detail=<cfc_app.bill_detail.BillDetail object at 0x7fbd3ab079e8> 05-26 21:01:23 extract_files:process_detail [DEBUG] 366:Creating LAW record: OH-HB0001-1815-Y2021 05-26 21:01:23 extract_files:process_bill [DEBUG] 345:bill_name=OH-HB0001-1815-Y2021.pdf bill_hash=None 05-26 21:01:23 extract_files:process_location [ERROR] 238: Process Error. HTTPSConnectionPool(host='search-prod.lis.state.oh.us', port=443): Max retries exceeded with url: /solarapi/v1/general_assembly_134/bills/hb1/IN/00?format=pdf (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)'),))

tonypearson commented 3 years ago

We can try verify=False on requests.get() but see this: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings

tonypearson commented 3 years ago

/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/urllib3/connectionpool.py:1020: InsecureRequestWarning: Unverified HTTPS request is being made to host 'search-prod.lis.state.oh.us'. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warnings

tonypearson commented 3 years ago

More errors: (Legit-Info) [Yoga Legit-Info]$ ./stage1 qcluster Using SQLite3 16:17:31 [Q] INFO Q Cluster single-low-kilo-network starting. 16:17:31 [Q] INFO Process-1:1 ready for work at 18895 16:17:31 [Q] INFO Process-1:2 ready for work at 18896 16:17:31 [Q] INFO Process-1:3 ready for work at 18897 16:17:31 [Q] INFO Process-1:4 ready for work at 18898 16:17:31 [Q] INFO Process-1:5 monitoring at 18899 16:17:31 [Q] INFO Process-1 guarding cluster single-low-kilo-network 16:17:31 [Q] INFO Process-1:6 pushing tasks at 18900 16:17:31 [Q] INFO Q Cluster single-low-kilo-network running. 16:17:31 [Q] INFO Process-1:1 processing [cat-uniform-louisiana-floor] 05-27 16:17:31 tasks:b_extract_files [INFO] 63:task started: b_extract_files 16:17:31 [Q] INFO Process-1:2 processing [queen-bravo-five-neptune] 05-27 16:17:31 tasks:b_extract_files [INFO] 63:task started: b_extract_files 16:17:31 [Q] INFO Process-1:3 processing [foxtrot-connecticut-equal-oscar] 05-27 16:17:31 tasks:b_extract_files [INFO] 63:task started: b_extract_files [nltk_data] Downloading package punkt to /home/tpearson/nltk_data... [nltk_data] Downloading package punkt to /home/tpearson/nltk_data... [nltk_data] Downloading package punkt to /home/tpearson/nltk_data... [nltk_data] Package punkt is already up-to-date! [nltk_data] Package punkt is already up-to-date! [nltk_data] Package punkt is already up-to-date! 05-27 16:17:34 log_time:time_now [INFO] Starting extract_files at May-27 09:17AM MST 05-27 16:17:34 log_time:time_now [INFO] Starting extract_files at May-27 09:17AM MST 05-27 16:17:34 log_time:time_now [INFO] Starting extract_files at May-27 09:17AM MST 05-27 16:17:34 extract_files:handle [INFO] 194:Processing: United States (US) 05-27 16:17:34 extract_files:handle [INFO] 194:Processing: United States (US) 05-27 16:17:34 extract_files:handle [INFO] 194:Processing: United States (US) 05-27 16:19:41 extract_files:write_file [INFO] 478:Writing: US-HB0736-1823-Y2021.txt 05-27 16:19:43 extract_files:write_file [INFO] 478:Writing: US-HB0736-1823-Y2021.txt 05-27 16:19:44 extract_files:process_location [ERROR] 238: Process Error. UNIQUE constraint failed: cfc_app_hash.item_name, cfc_app_hash.fob_method Traceback (most recent call last): File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 413, in execute return Database.Cursor.execute(self, query, params) sqlite3.IntegrityError: UNIQUE constraint failed: cfc_app_hash.item_name, cfc_app_hash.fob_method

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 186, in process_location self.process_json(json_name) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 256, in process_json self.process_zip(msg_bytes) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 282, in process_zip processed = self.process_source(json_data) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 326, in process_source processed = self.skip_if_exists(text_name, detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 383, in skip_if_exists processed = self.process_bill(detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 427, in process_bill save_source_hash(bill_hash, detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/models.py", line 308, in save_source_hash bill_hash.save() File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 754, in save force_update=force_update, update_fields=update_fields) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 792, in save_base force_update, using, update_fields, File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 895, in _save_table results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 935, in _do_insert using=using, raw=raw, File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/query.py", line 1254, in _insert return query.get_compiler(using=using).execute_sql(returning_fields) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1397, in execute_sql cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 98, in execute return super().execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 66, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers return executor(sql, params, many, context) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/utils.py", line 90, in exit raise dj_exc_value.with_traceback(traceback) from exc_value File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 413, in execute return Database.Cursor.execute(self, query, params) django.db.utils.IntegrityError: UNIQUE constraint failed: cfc_app_hash.item_name, cfc_app_hash.fob_method 16:19:44 [Q] ERROR Failed [queen-bravo-five-neptune] - 238: Process Error. UNIQUE constraint failed: cfc_app_hash.item_name, cfc_app_hash.fob_method : Traceback (most recent call last): File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 413, in execute return Database.Cursor.execute(self, query, params) sqlite3.IntegrityError: UNIQUE constraint failed: cfc_app_hash.item_name, cfc_app_hash.fob_method

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 186, in process_location self.process_json(json_name) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 256, in process_json self.process_zip(msg_bytes) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 282, in process_zip processed = self.process_source(json_data) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 326, in process_source processed = self.skip_if_exists(text_name, detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 383, in skip_if_exists processed = self.process_bill(detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 427, in process_bill save_source_hash(bill_hash, detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/models.py", line 308, in save_source_hash bill_hash.save() File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 754, in save force_update=force_update, update_fields=update_fields) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 792, in save_base force_update, using, update_fields, File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 895, in _save_table results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 935, in _do_insert using=using, raw=raw, File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/query.py", line 1254, in _insert return query.get_compiler(using=using).execute_sql(returning_fields) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1397, in execute_sql cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 98, in execute return super().execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 66, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers return executor(sql, params, many, context) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/utils.py", line 90, in exit raise dj_exc_value.with_traceback(traceback) from exc_value File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 413, in execute return Database.Cursor.execute(self, query, params) django.db.utils.IntegrityError: UNIQUE constraint failed: cfc_app_hash.item_name, cfc_app_hash.fob_method

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django_q/cluster.py", line 436, in worker res = f(*task["args"], task["kwargs"]) File "/home/tpearson/Develop/Legit-Info/cfc_app/tasks.py", line 69, in b_extract_files call_command(cmd, '--api', '--skip') File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/core/management/init.py", line 168, in call_command return command.execute(*args, *defaults) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/core/management/base.py", line 371, in execute output = self.handle(args, options) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 144, in handle self.process_location(state) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 190, in process_location raise ExtractTextError(err_msg) from exc cfc_app.management.commands.extract_files.ExtractTextError: 238: Process Error. UNIQUE constraint failed: cfc_app_hash.item_name, cfc_app_hash.fob_method

05-27 16:19:46 extract_files:write_file [INFO] 478:Writing: US-HB0738-1823-Y2021.txt 05-27 16:19:48 extract_files:write_file [INFO] 478:Writing: US-HB0738-1823-Y2021.txt 05-27 16:19:48 extract_files:process_location [ERROR] 238: Process Error. UNIQUE constraint failed: cfc_app_hash.item_name, cfc_app_hash.fob_method Traceback (most recent call last): File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 413, in execute return Database.Cursor.execute(self, query, params) sqlite3.IntegrityError: UNIQUE constraint failed: cfc_app_hash.item_name, cfc_app_hash.fob_method

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 186, in process_location self.process_json(json_name) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 256, in process_json self.process_zip(msg_bytes) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 282, in process_zip processed = self.process_source(json_data) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 326, in process_source processed = self.skip_if_exists(text_name, detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 383, in skip_if_exists processed = self.process_bill(detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 427, in process_bill save_source_hash(bill_hash, detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/models.py", line 308, in save_source_hash bill_hash.save() File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 754, in save force_update=force_update, update_fields=update_fields) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 792, in save_base force_update, using, update_fields, File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 895, in _save_table results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 935, in _do_insert using=using, raw=raw, File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/query.py", line 1254, in _insert return query.get_compiler(using=using).execute_sql(returning_fields) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1397, in execute_sql cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 98, in execute return super().execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 66, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers return executor(sql, params, many, context) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/utils.py", line 90, in exit raise dj_exc_value.with_traceback(traceback) from exc_value File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 413, in execute return Database.Cursor.execute(self, query, params) django.db.utils.IntegrityError: UNIQUE constraint failed: cfc_app_hash.item_name, cfc_app_hash.fob_method 16:19:48 [Q] ERROR Failed [cat-uniform-louisiana-floor] - 238: Process Error. UNIQUE constraint failed: cfc_app_hash.item_name, cfc_app_hash.fob_method : Traceback (most recent call last): File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 413, in execute return Database.Cursor.execute(self, query, params) sqlite3.IntegrityError: UNIQUE constraint failed: cfc_app_hash.item_name, cfc_app_hash.fob_method

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 186, in process_location self.process_json(json_name) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 256, in process_json self.process_zip(msg_bytes) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 282, in process_zip processed = self.process_source(json_data) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 326, in process_source processed = self.skip_if_exists(text_name, detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 383, in skip_if_exists processed = self.process_bill(detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 427, in process_bill save_source_hash(bill_hash, detail) File "/home/tpearson/Develop/Legit-Info/cfc_app/models.py", line 308, in save_source_hash bill_hash.save() File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 754, in save force_update=force_update, update_fields=update_fields) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 792, in save_base force_update, using, update_fields, File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 895, in _save_table results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/base.py", line 935, in _do_insert using=using, raw=raw, File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/query.py", line 1254, in _insert return query.get_compiler(using=using).execute_sql(returning_fields) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/sql/compiler.py", line 1397, in execute_sql cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 98, in execute return super().execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 66, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers return executor(sql, params, many, context) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/utils.py", line 90, in exit raise dj_exc_value.with_traceback(traceback) from exc_value File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/utils.py", line 84, in _execute return self.cursor.execute(sql, params) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/backends/sqlite3/base.py", line 413, in execute return Database.Cursor.execute(self, query, params) django.db.utils.IntegrityError: UNIQUE constraint failed: cfc_app_hash.item_name, cfc_app_hash.fob_method

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django_q/cluster.py", line 436, in worker res = f(*task["args"], task["kwargs"]) File "/home/tpearson/Develop/Legit-Info/cfc_app/tasks.py", line 69, in b_extract_files call_command(cmd, '--api', '--skip') File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/core/management/init.py", line 168, in call_command return command.execute(*args, *defaults) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/core/management/base.py", line 371, in execute output = self.handle(args, options) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 144, in handle self.process_location(state) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/extract_files.py", line 190, in process_location raise ExtractTextError(err_msg) from exc cfc_app.management.commands.extract_files.ExtractTextError: 238: Process Error. UNIQUE constraint failed: cfc_app_hash.item_name, cfc_app_hash.fob_method

05-27 16:19:50 extract_files:write_file [INFO] 478:Writing: US-HB0740-1823-Y2021.txt 05-27 16:19:55 extract_files:write_file [INFO] 478:Writing: US-HB0741-1823-Y2021.txt 05-27 16:19:59 extract_files:write_file [INFO] 478:Writing: US-HB0742-1823-Y2021.txt 16:20:01 [Q] INFO Enqueued 36 16:20:01 [Q] INFO Process-1 created a task from schedule [c_analyze_text] 16:20:01 [Q] INFO Process-1:4 processing [magazine-dakota-lithium-low] 05-27 16:20:01 tasks:c_analyze_text [INFO] 76:task started: c_analyze_text 05-27 16:20:02 extract_files:write_file [INFO] 478:Writing: US-HB0743-1823-Y2021.txt 05-27 16:20:03 log_time:time_now [INFO] Starting analyze_text at May-27 09:20AM MST 16:20:03 [Q] ERROR Failed [magazine-dakota-lithium-low] - Cannot resolve keyword 'text' into field. Choices are: criteria, date_added, id, iname, laws, profile : Traceback (most recent call last): File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django_q/cluster.py", line 436, in worker res = f(*task["args"], task["kwargs"]) File "/home/tpearson/Develop/Legit-Info/cfc_app/tasks.py", line 82, in c_analyze_text call_command(cmd, '--api', '--skip', '--compare') File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/core/management/init.py", line 168, in call_command return command.execute(*args, *defaults) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/core/management/base.py", line 371, in execute output = self.handle(args, options) File "/home/tpearson/Develop/Legit-Info/cfc_app/management/commands/analyze_text.py", line 138, in handle impacts = Impact.objects.all().exclude(text='None') File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/query.py", line 950, in exclude return self._filter_or_exclude(True, *args, kwargs) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/query.py", line 962, in _filter_or_exclude clone._filter_or_exclude_inplace(negate, *args, *kwargs) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/query.py", line 967, in _filter_or_exclude_inplace self._query.add_q(~Q(args, kwargs)) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/sql/query.py", line 1358, in addq clause, = self._add_q(q_object, self.used_aliases) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/sql/query.py", line 1380, in _add_q split_subq=split_subq, check_filterable=check_filterable, File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/sql/query.py", line 1258, in build_filter lookups, parts, reffed_expression = self.solve_lookup_type(arg) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/sql/query.py", line 1084, in solve_lookuptype , field, _, lookup_parts = self.names_to_path(lookup_splitted, self.get_meta()) File "/home/tpearson/.local/share/virtualenvs/Legit-Info-YLTR15_m/lib/python3.6/site-packages/django/db/models/sql/query.py", line 1482, in names_to_path "Choices are: %s" % (name, ", ".join(available))) django.core.exceptions.FieldError: Cannot resolve keyword 'text' into field. Choices are: criteria, date_added, id, iname, laws, profile

05-27 16:20:07 extract_files:write_file [INFO] 478:Writing: US-HB0744-1823-Y2021.txt 05-27 16:20:10 extract_files:write_file [INFO] 478:Writing: US-HB0745-1823-Y2021.txt 05-27 16:20:14 extract_files:write_file [INFO] 478:Writing: US-HB0746-1823-Y2021.txt 05-27 16:20:18 extract_files:write_file [INFO] 478:Writing: US-HB0748-1823-Y2021.txt 05-27 16:20:18 extract_files:handle [INFO] 194:Processing: Arizona (AZ)

This appears to be a qcluster quirk. If you specify "Once" and repeat > 1, then it kicks off multiple concurrent tasks.

upkarlidder commented 3 years ago

Not sure if this was captured. Error in the CE job legit-info-job-extractfiles-jobrun-cbhj8

Jun 12 09:07:01 Code Engine legit-info-job-extractfiles-jobrun-cbhj8-0-0 >.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>.>..
Jun 12 09:07:01 Code Engine legit-info-job-extractfiles-jobrun-cbhj8-0-0 06-12 16:06:57 extract_files:process_location [ERROR] 238: Process Error. server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
Jun 12 09:07:01 Code Engine legit-info-job-extractfiles-jobrun-cbhj8-0-0 Traceback (most recent call last):
  File "/opt/app-root/lib64/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
Jun 12 09:07:01 Code Engine legit-info-job-extractfiles-jobrun-cbhj8-0-0 psycopg2.OperationalError: server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
Jun 12 09:07:01 Code Engine legit-info-job-extractfiles-jobrun-cbhj8-0-0 The above exception was the direct cause of the following exception:
Jun 12 09:07:01 Code Engine legit-info-job-extractfiles-jobrun-cbhj8-0-0 Traceback (most recent call last):
  File "/opt/app-root/src/cfc_app/management/commands/extract_files.py", line 186, in process_location
    self.process_json(json_name)
  File "/opt/app-root/src/cfc_app/management/commands/extract_files.py", line 227, in process_json
    source_hash = Hash.find_item_name(json_name)
  File "/opt/app-root/src/cfc_app/models.py", line 288, in find_item_name
    record = Hash.objects.get(item_name=name,
  File "/opt/app-root/lib64/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
  File "/opt/app-root/lib64/python3.8/site-packages/django/db/models/query.py", line 425, in get
    num = len(clone)
  File "/opt/app-root/lib64/python3.8/site-packages/django/db/models/query.py", line 269, in __len__
    self._fetch_all()
  File "/opt/app-root/lib64/python3.8/site-packages/django/db/models/query.py", line 1308, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/opt/app-root/lib64/python3.8/site-packages/django/db/models/query.py", line 53, in __iter__
    results = compiler.execute_sql(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size)
  File "/opt/app-root/lib64/python3.8/site-packages/django/db/models/sql/compiler.py", line 1156, in execute_sql
    cursor.execute(sql, params)
  File "/opt/app-root/lib64/python3.8/site-packages/django/db/backends/utils.py", line 98, in execute
    return super().execute(sql, params)
  File "/opt/app-root/lib64/python3.8/site-packages/django/db/backends/utils.py", line 66, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/opt/app-root/lib64/python3.8/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/opt/app-root/lib64/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
  File "/opt/app-root/lib64/python3.8/site-packages/django/db/utils.py", line 90, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/opt/app-root/lib64/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute
    return self.cursor.execute(sql, params)
Jun 12 09:07:01 Code Engine legit-info-job-extractfiles-jobrun-cbhj8-0-0 django.db.utils.OperationalError: server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
Jun 12 09:07:01 Code Engine legit-info-job-extractfiles-jobrun-cbhj8-0-0 ExtractTextError: 238: Process Error. server closed the connection unexpectedly
    This probably means the server terminated abnormally
    before or while processing the request.
github-actions[bot] commented 3 years ago

:wave: Hi! This issue has been marked stale due to inactivity. If no further activity occurs, it will automatically be closed.