Closed n10000k closed 1 year ago
Sorry I'm not familiar with this setup. Maybe anyone else can help you...
What is the server error that you get?
Seeing the same issue on my end after upgrading from 3.9.2
to 4.6.0
.
{"error":"Internal server error","message":"EXECABORT Transaction discarded because of previous errors.","details":"ReplyError: EXECABORT Transaction discarded because of previous errors.\n at parseError (/var/app/current/node_modules/redis-parser/lib/parser.js:179:12)\n at parseType (/var/app/current/node_modules/redis-parser/lib/parser.js:302:14)"}
Details:
Memory usage
1.1 GB of 1.1 GB100.00%
Peak memory usage
1.11 GB
Fragmentation ratio
0.82
Connected clients
407
Blocked clients
51
Version
6.0.5
Mode
standalone
OS
Amazon ElastiCache
Up time
30 days, 2 hours, 26 minutes
Interesting to note that this doesn't always happen. It seems to happen in waves where 3-4 requests would succeed and the UI will load but then 10 next requests would fail with Internal Server Error and then back to 3-4 successful requests.
I can confirm the same happens still to me, just randomly (not sure on the amount of requests). It brings up the queues then randomly has some form of timeout or something going on where it throws the above message. Maybe some form of issue with ElastiCache?
Disregard my previous message. As it's evident from the details I posted, the issue comes from the server being out of memory. I just noticed the OOM issues in my server logs.
Once I upgraded the cache to a bigger instance, the errors went away.
@n10000k ?
We run it in cluster mode, no memory issues. I feel like after x amount of polling within a cluster mode it drops off. I can share an example to replicate locally later on.
I'll happy to get a replication example
This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.
API Uri -
/api/queues?activeQueue=&page=1&jobsPerPage=10
Error:
Local setup runs Redis cluster and works fine, same queue setup using key prefixes. Has anyone had this issue when deploying to an instance on AWS? Are there any better ways to debug this? thanks in advance