Closed brianoflondon closed 2 years ago
This PR doesn't make any code changes, what am I missing?
Oh. You already pushed the code change to develop
. Please don't do that.
Oh. You already pushed the code change to
develop
. Please don't do that.
I kinda messed up and tried to reverse that commit and then put my changes in a branch.
Success... this is the only error I've had for days and the code dealt with it properly:
2022-08-13T13:02:55+0000 | INFO | Lighthive Fastest: https://rpc.ecency.com
2022-08-13T13:02:55+0000 | INFO | Status - Uptime: 7:02:49 | IRIs Received: 46 | IRIs Deduped: 46 | IRIs Sent: 46 | last_node: https://rpc.ecency.com
2022-08-13T13:04:59+0000 | INFO | IRI batch_id 2f810e23-3cd3-43ee-8d20-0254a320f525 - Size of IRIs: 36
2022-08-13T13:04:59+0000 | INFO | Received 1 IRIs
2022-08-13T13:05:03+0000 | INFO | Backing off _send_request(...) for 0.9s (requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='rpc.ecency.com', port=443): Read timed out. (read timeout=3))
2022-08-13T13:05:40+0000 | INFO | Backing off _send_request(...) for 0.1s (requests.exceptions.ReadTimeout: HTTPSConnectionPool(host='rpc.ecency.com', port=443): Read timed out. (read timeout=3))
2022-08-13T13:05:41+0000 | ERROR | send_notification error: transaction expiration exception:
2022-08-13T13:05:41+0000 | ERROR | Failed to send 1 IRIs
Traceback (most recent call last):
File "/home/podping/app/src/podping_hivewriter/cli/podping.py", line 245, in server
loop = asyncio.get_running_loop()
RuntimeError: no running event loop
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/podping/app/src/podping_hivewriter/podping_hivewriter.py", line 482, in failure_retry
await self.send_notification_iris(
File "/home/podping/app/src/podping_hivewriter/podping_hivewriter.py", line 456, in send_notification_iris
await self.send_notification(payload.dict(), hive_operation_id)
File "/home/podping/app/src/podping_hivewriter/podping_hivewriter.py", line 416, in send_notification
raise ex
File "/home/podping/app/src/podping_hivewriter/podping_hivewriter.py", line 401, in send_notification
await self._async_hive_broadcast(op=op, dry_run=self.dry_run)
File "/home/podping/app/src/podping_hivewriter/async_wrapper.py", line 32, in wrapper
return await async_fn(*args, **kwargs)
File "/home/podping/app/.venv/lib/python3.10/site-packages/asgiref/sync.py", line 435, in __call__
ret = await asyncio.wait_for(future, timeout=None)
File "/usr/local/lib/python3.10/asyncio/tasks.py", line 408, in wait_for
return await fut
File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/podping/app/.venv/lib/python3.10/site-packages/asgiref/sync.py", line 474, in thread_handler
return func(*args, **kwargs)
File "/home/podping/app/.venv/lib/python3.10/site-packages/lighthive/client.py", line 184, in broadcast
return self.transaction_builder.broadcast(
File "/home/podping/app/.venv/lib/python3.10/site-packages/lighthive/broadcast/transaction_builder.py", line 202, in broadcast
resp = self.client('condenser_api').broadcast_transaction(
File "/home/podping/app/.venv/lib/python3.10/site-packages/lighthive/client.py", line 53, in callable
return self.request(attr, *args, **kwargs)
File "/home/podping/app/.venv/lib/python3.10/site-packages/lighthive/client.py", line 152, in request
self.validate_response(response)
File "/home/podping/app/.venv/lib/python3.10/site-packages/lighthive/client.py", line 164, in validate_response
raise RPCNodeException(
lighthive.exceptions.RPCNodeException: transaction expiration exception:
2022-08-13T13:05:41+0000 | ERROR | transaction expiration exception:
Traceback (most recent call last):
File "/home/podping/app/src/podping_hivewriter/cli/podping.py", line 245, in server
loop = asyncio.get_running_loop()
RuntimeError: no running event loop
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/podping/app/src/podping_hivewriter/podping_hivewriter.py", line 482, in failure_retry
await self.send_notification_iris(
File "/home/podping/app/src/podping_hivewriter/podping_hivewriter.py", line 456, in send_notification_iris
await self.send_notification(payload.dict(), hive_operation_id)
File "/home/podping/app/src/podping_hivewriter/podping_hivewriter.py", line 416, in send_notification
raise ex
File "/home/podping/app/src/podping_hivewriter/podping_hivewriter.py", line 401, in send_notification
await self._async_hive_broadcast(op=op, dry_run=self.dry_run)
File "/home/podping/app/src/podping_hivewriter/async_wrapper.py", line 32, in wrapper
return await async_fn(*args, **kwargs)
File "/home/podping/app/.venv/lib/python3.10/site-packages/asgiref/sync.py", line 435, in __call__
ret = await asyncio.wait_for(future, timeout=None)
File "/usr/local/lib/python3.10/asyncio/tasks.py", line 408, in wait_for
return await fut
File "/usr/local/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/podping/app/.venv/lib/python3.10/site-packages/asgiref/sync.py", line 474, in thread_handler
return func(*args, **kwargs)
File "/home/podping/app/.venv/lib/python3.10/site-packages/lighthive/client.py", line 184, in broadcast
return self.transaction_builder.broadcast(
File "/home/podping/app/.venv/lib/python3.10/site-packages/lighthive/broadcast/transaction_builder.py", line 202, in broadcast
resp = self.client('condenser_api').broadcast_transaction(
File "/home/podping/app/.venv/lib/python3.10/site-packages/lighthive/client.py", line 53, in callable
return self.request(attr, *args, **kwargs)
File "/home/podping/app/.venv/lib/python3.10/site-packages/lighthive/client.py", line 152, in request
self.validate_response(response)
File "/home/podping/app/.venv/lib/python3.10/site-packages/lighthive/client.py", line 164, in validate_response
raise RPCNodeException(
lighthive.exceptions.RPCNodeException: transaction expiration exception:
2022-08-13T13:05:41+0000 | WARNING | Waiting 3s before retry
2022-08-13T13:05:44+0000 | INFO | FAILURE COUNT: 1 - RETRYING 1 IRIs
2022-08-13T13:05:55+0000 | INFO | Lighthive Node: https://rpc.ausbit.dev
2022-08-13T13:05:55+0000 | INFO | JSON size: 96
2022-08-13T13:05:55+0000 | INFO | FAILURE CLEARED after 1 retries, 3s
If the exception is being caught and handled gracefully, is there any way to clean that up and make it less of a huge log barf?
If the exception is being caught and handled gracefully, is there any way to clean that up and make it less of a huge log barf?
I think we can stop logging known exception tracebacks now that we know what's working.
@agates I monitor my logs and I think something I fixed before... needed more fixing.