Cassandra Reaper logs posting warn messages regarding remembered principals and RememberMeManager

[vchauhan81]

Project board link

We are using reaper version 3.2.0 and one of our customer is seeing following errors in reaper logs. However, on reaper UI, it's showing that all reapir are completed successfully for all schema. Is this something we need to worry about ?

ERROR [2023-04-24 05:29:41,405] [SchedulingManagerTimer] i.c.s.SchedulingManager - Failed managing repair schedule with id 'f28c4b40-d913-11ed-a539-13326efc81b2' java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask4fd1b0eb rejected from java.util.concurrent.ScheduledThreadPoolExecutor@d883e0cb[Shutting down, pool size = 15, active threads = 0, queued tasks = 5, completed tasks = 402502] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2074) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:841) at java.util.concurrent.ScheduledThreadPoolExecutor.delayedExecute(ScheduledThreadPoolExecutor.java:337) at java.util.concurrent.ScheduledThreadPoolExecutor.schedule(ScheduledThreadPoolExecutor.java:544)

ERROR [2023-04-24 05:29:41,462] [mydatabasecluster:24c285e0-e281-11ed-804e-095add68cf60] i.c.s.RepairRunner - RepairRun FAILURE, scheduling retry java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask7a368341 rejected from java.util.concurrent.ScheduledThreadPoolExecutor@d883e0cb[Shutting down, pool size = 15, active threads = 1, queued tasks = 4, completed tasks = 402502] at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2074) at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:841)

WARN [2023-04-24 05:30:07,000] [mydatabase Cluster-nio-worker-2] c.d.d.c.RequestHandler - Query '[2 bound values] SELECT FROM repair_run_by_cluster_v2 WHERE cluster_name = ? limit ?;' generated server side warning(s): Read 307 live rows and 2070 tombstone cells for query SELECT FROM reaper_current.repair_run_by_cluster_v2 WHERE cluster_name = mydatabasecluster LIMIT 1000; token -6675551992983735733 (see tombstone_warn_threshold) WARN [2023-04-24 05:30:07,037] [mydatabase Cluster-nio-worker-2] c.d.d.c.RequestHandler - Query '[2 bound values] SELECT FROM repair_run_by_cluster_v2 WHERE cluster_name = ? limit ?;' generated server side warning(s): Read 307 live rows and 2070 tombstone cells for query SELECT FROM reaper_current.repair_run_by_cluster_v2 WHERE cluster_name = mydatabasecluster LIMIT 1000; token -6675551992983735733 (see tombstone_warn_threshold)

. WARN [2023-04-24 05:30:07,215] [main] i.c.ReaperApplication - Reaper is ready to get things done!

. WARN [2023-04-24 05:30:07,239] [mydatabase Cluster-nio-worker-2] c.d.d.c.RequestHandler - Query '[2 bound values] SELECT FROM repair_run_by_cluster_v2 WHERE cluster_name = ? limit ?;' generated server side warning(s): Read 307 live rows and 2070 tombstone cells for query SELECT FROM reaper_current.repair_run_by_cluster_v2 WHERE cluster_name = mydatabasecluster LIMIT 1000; token -6675551992983735733 (see tombstone_warn_threshold)

WARN [2023-04-24 08:57:42,054] [dw-135] o.e.j.h.HttpParser - Header is too large 8193>8192 WARN [2023-04-24 09:02:01,769] [dw-109] o.a.s.m.AbstractRememberMeManager - There was a failure while trying to retrieve remembered principals. This could be due to a configuration problem or corrupted principals. This could also be due to a recently changed encryption key, if you are using a shiro.ini file, this property would be 'securityManager.rememberMeManager.cipherKey' see: http://shiro.apache.org/web.html#Web-RememberMeServices. The remembered identity will be forgotten and not used for this request. WARN [2023-04-24 09:02:01,771] [dw-109] o.a.s.m.DefaultSecurityManager - Delegate RememberMeManager instance of type [org.apache.shiro.web.mgt.CookieRememberMeManager] threw an exception during getRememberedPrincipals(). org.apache.shiro.crypto.CryptoException: Unable to execute 'doFinal' with cipher instance [javax.crypto.Cipher@b9d39294]. at org.apache.shiro.crypto.JcaCipherService.crypt(JcaCipherService.java:462) at org.apache.shiro.crypto.JcaCipherService.crypt(JcaCipherService.java:445) at org.apache.shiro.crypto.JcaCipherService.decrypt(JcaCipherService.java:390) at org.apache.shiro.crypto.JcaCipherService.decrypt(JcaCipherService.java:382) at org.apache.shiro.mgt.AbstractRememberMeManager.decrypt(AbstractRememberMeManager.java:482) .

┆Issue is synchronized with this Jira Story by Unito ┆Issue Number: REAP-61

[adejanovski]

Hi @vchauhan81,

apparently we're hitting this. No big deal, I guess that's something we can fix using the suggested method in the linked ticket.

[vchauhan81]

Hi @adejanovski Thanks for prompt response. Any idea, in which version we can expect the fix ?

[vchauhan81]

@adejanovski Any idea, in which version fix is being planned ? Thanks in advance.