Closed spawn-guy closed 4 days ago
BTW, for the aio_pool to work on py 3.8 i had to modify typing (Tuple
and Dict
) and add a backport version of aio.to_thread
- otherwise works decent(no concurrency support) as a drop-in replacement for kai's celery-pool-asyncio
i've debugged the code a bit yesterday. and now i have a few options:
sts-assume-role
, but i already assume the same role from current ec2-instance-profile
. code should understand that credentials are "refreshable/expiring", as the boto3
/kombu
doesn't understand this now boto3
will understand that the credentials from ec2 instance-profile
are "expiring" and need to be "refreshed" every now-and-then see https://github.com/boto/boto3/issues/443 like in Java/Ruby/PHP SDKwhat are your thoughts, community?
something changed and it is working now. moving on
started seeing these messages yesterday. and yesterday i was on
celery<5
(due to async support)today i migrated to celery v5 with a modified version of https://github.com/the-wondersmith/celery-aio-pool (to support our current py3.8)
celery==5.3.6 kombu==5.3.5
and i still see this message. a good thing that celery v5 worker restarts afterwards (unlike v4 when it was hanging indefinitely)
processes i am running can take a long time to finish. as a mitigation technique i tried increasing message visibility on SQS side to 30m (from 20m) - it is better now, but still i see this message