Closed ConnorL33t closed 4 years ago
Hmmm. Ok. I can't get much from that error message, but it does tell me ways that I can improve that error message, i.e. track why a connection has become unusable and report it. Will hack something in but: it might need one round of better logging to know where to look for the actual fix.
On Thu, 11 Apr 2019, 19:25 Connor Lee, notifications@github.com wrote:
Hi.
Recently updated our server with StackExchange.Redis v2.0.600. About a day or two into running the library I started receiving thousands of these errors
RedisConnectionException: No connection is available to service this operation; It was not possible to connect to the redis server(s). To create a disconnected multiplexer, disable AbortOnConnectFail. ConnectTimeout; IOCP: (Busy=45,Free=955,Min=1000,Max=1000), WORKER: (Busy=43,Free=32724,Min=100,Max=32767), Local-CPU: n/a
Immediate issues were resolved by a web app restart, shortly after I reverted back to an older version of StackExchange.Redis.
Running Azure Redis Cache, ~20 client connections during these errors, ~120KiB throughput. 100Mb cache size. Redis Resource health shows "good"
— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/StackExchange/StackExchange.Redis/issues/1120, or mute the thread https://github.com/notifications/unsubscribe-auth/AABDsCoxWZnlo02E625T8zPhdi7Giky9ks5vf34AgaJpZM4cqjO7 .
I understand, I'll update and try to get a better error message when it's on nuget.
We are also getting these errors on Azure Redis. Application restart fixes them temporarily, but then they are starting again. We are on 2.0.571
We're also seeing this error pop up recently with the v2 library. We have to restart our services in order to mitigate the issue, but it seems to come back after an hour or so.
Did anyone make any progress on figuring out what's going wrong?
Same error. Turning on/off the AbortOnConnectFail setting has no impact. Any ideas what is the main issue here ?
Possibly related in 2.0.601.3402 AWS ElasticCache Redis Version 2.8.23 1,100 connections during this time with a 64 GB cache and ~35MB throughput Process gets in this state and requires a restart
RedisConnectionException: No connection is available to service this operation: SETEX ****;
SocketClosed (ReadEndOfStream, last-recv: 0) on ***:6379/Interactive, Idle/MarkProcessed,
last: HSET, origin: ReadFromPipe, outstanding: 0, last-read: 0s ago, last-write: 46s ago, keep-alive: 60s,
state: ConnectedEstablished, mgr: 9 of 10 available, in: 0, in-pipe: 0, out-pipe: 0, last-heartbeat: 0s ago,
last-mbeat: 0s ago, global: 0s ago, v: 2.0.601.3402; IOCP: (Busy=0,Free=1000,Min=4,Max=1000),
WORKER: (Busy=3,Free=32764,Min=4,Max=32767), Local-CPU: n/a
Any update on the issue?
We are also experiencing similar exceptions after updating to v2
StackExchange.Redis.RedisConnectionException: No connection is available to service this operation: HMSET ...; SocketClosed on .../Interactive, Idle/MarkProcessed, last: HMSET, origin: ReadFromPipe, outstanding: 0, last-read: 0s ago, last-write: 55s ago, keep-alive: 60s, state: ConnectedEstablished, mgr: 9 of 10 available, in: 0, last-heartbeat: 0s ago, last-mbeat: 0s ago, global: 0s ago, v: 2.0.601.3402; IOCP: (Busy=1,Free=999,Min=200,Max=1000), WORKER: (Busy=5,Free=32762,Min=200,Max=32767), Local-CPU: n/a
We are also experiencing these quite regularly in our Azure Cloud Services. Using version 2.0.571.
Today we downgraded our dll to 2.0.5 and we are not experiencing those errors anymore
@Styrna what version specifically did you go to? As noted, we are seeing it with 2.0.571
but I see versions 2.0.505
, 2.0.513
and 2.0.519
out there. Also there are some later versions 2.0.588
and 2.0.593
.
2.0.519.65453 is working fine for us.
Same issue for me on 2.0.601
No connection is available to service this operation: GET ...; An existing connection was forcibly closed by the remote host; IOCP: (Busy=1,Free=999,Min=2,Max=1000), WORKER: (Busy=3,Free=8188,Min=2,Max=8191), Local-CPU: n/a SocketFailure on ..., Idle/Faulted, last: GET, origin: ReadFromPipe, outstanding: 10, last-read: 0s ago, last-write: 0s ago, keep-alive: 60s, state: ConnectedEstablished, mgr: 9 of 10 available, in: 0, last-heartbeat: 0s ago, last-mbeat: 0s ago, global: 0s ago, v: 2.0.601.3402 Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. An existing connection was forcibly closed by the remote host
Not sure if this helps:
StackExchange.Redis.RedisConnectionException:
...
System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable`1+ConfiguredValueTaskAwaiter.GetResult (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Net.Security.SslStreamInternal+<<FillBufferAsync>g__InternalFillBufferAsync|38_0>d`1.MoveNext (System.Net.Security, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.ConfiguredValueTaskAwaitable`1+ConfiguredValueTaskAwaiter.GetResult (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Net.Security.SslStreamInternal+<ReadAsyncInternal>d__34`1.MoveNext (System.Net.Security, Version=4.1.1.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at Pipelines.Sockets.Unofficial.StreamConnection+AsyncStreamPipe+<CopyFromStreamToReadPipe>d__13.MoveNext (Pipelines.Sockets.Unofficial, Version=1.0.0.0, Culture=neutral, PublicKeyToken=42ea0a778e13fbe2Pipelines.Sockets.Unofficial, Version=1.0.0.0, Culture=neutral, PublicKeyToken=42ea0a778e13fbe2: C:\Code\Pipelines.Sockets.Unofficial\src\Pipelines.Sockets.Unofficial\StreamConnection.AsyncStreamPipe.csPipelines.Sockets.Unofficial, Version=1.0.0.0, Culture=neutral, PublicKeyToken=42ea0a778e13fbe2: 97)
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw (System.Private.CoreLib, Version=4.0.0.0, Culture=neutral, PublicKeyToken=7cec85d7bea7798e)
at System.IO.Pipelines.PipeCompletion.ThrowLatchedException (System.IO.Pipelines, Version=4.0.0.1, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51)
at System.IO.Pipelines.Pipe.GetReadResult (System.IO.Pipelines, Version=4.0.0.1, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51)
at System.IO.Pipelines.Pipe.GetReadAsyncResult (System.IO.Pipelines, Version=4.0.0.1, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51)
at System.IO.Pipelines.Pipe+DefaultPipeReader.GetResult (System.IO.Pipelines, Version=4.0.0.1, Culture=neutral, PublicKeyToken=cc7b13ffcd2ddd51)
at StackExchange.Redis.PhysicalConnection+<ReadFromPipe>d__110.MoveNext (StackExchange.Redis, Version=2.0.0.0, Culture=neutral, PublicKeyToken=c219ff1ca8c2ce46StackExchange.Redis, Version=2.0.0.0, Culture=neutral, PublicKeyToken=c219ff1ca8c2ce46: C:\projects\stackexchange-redis\src\StackExchange.Redis\PhysicalConnection.csStackExchange.Redis, Version=2.0.0.0, Culture=neutral, PublicKeyToken=c219ff1ca8c2ce46: 1475)
Inner exception System.Net.Sockets.SocketException handled at System.Net.Sockets.Socket+AwaitableSocketAsyncEventArgs.ThrowException:
Hi,
We've upgraded from 2.0.513 due to the performance issue, but now there is "No connection is available to service this operation" errors.
Any progress regarding this issue?
StackExchange.Redis.RedisConnectionException: No connection is available to service this operation: HSET MessageQueue:ProcessingListsData; IOCP: (Busy=0,Free=1000,Min=100,Max=1000), WORKER: (Busy=71,Free=1976,Min=100,Max=2047), Local-CPU: n/a at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor
1 processor, ServerEndPoint server)
at StackExchange.Redis.RedisBase.ExecuteSync[T](Message message, ResultProcessor1 processor, ServerEndPoint server) at StackExchange.Redis.RedisDatabase.HashSet(RedisKey key, RedisValue hashField, RedisValue value, When when, CommandFlags flags)
Hi,
We have a similar issue with 2.0.571. There are thousands of No connection is available to service this operation
in our Production environment (so it really affects Production). The issue could be resolved by restarting the service (node).
Running Azure Redis Cache, Client host: Azure VM. Clients and Server are located in the same region.
RedisConnectionException: Message: No connection is available to service this operation: XXX WWWWW; It was not possible to connect to the redis server(s). To create a disconnected multiplexer, disable AbortOnConnectFail. ConnectTimeout; IOCP: (Busy=0,Free=3000,Min=500,Max=3000), WORKER: (Busy=5,Free=2995,Min=500,Max=3000), Local-CPU: n/a
Hi,
We have a similar issue with 2.0.571. There are thousands of
No connection is available to service this operation
in our Production environment (so it really affects Production). The issue could be resolved by restarting the service (node).Running Azure Redis Cache, Client host: Azure VM. Clients and Server are located in the same region.
RedisConnectionException: Message: No connection is available to service this operation: XXX WWWWW; It was not possible to connect to the redis server(s). To create a disconnected multiplexer, disable AbortOnConnectFail. ConnectTimeout; IOCP: (Busy=0,Free=3000,Min=500,Max=3000), WORKER: (Busy=5,Free=2995,Min=500,Max=3000), Local-CPU: n/a
@mgravell This is causing a huge issue for our production servers and processing. We cannot take latest Reid’s package as it has severe performance issue and now we are experiencing this issue consistently. This is completely unacceptable - as according to you, it seems more of a logging issue - I understand that you need to log from exact location - as to why connections are being dropped, but still, this is not an acceptable behavior - specially when this package is recommended by MS Azure team. We have 8 different production instances and we are experiencing this issue on all production system. Out of all these instances, largest one is a shared deployment with 6 x 13GB Redis. Off course we are trying to handle all these connection issues in our pipeline, however, doing so is causing massive delays in the system. cc @MaximKojine
@tejasavora I think we're all frustrated, but remember this is OSS - @mgravell & StackOverflow are allowing us to use this library, and supporting our use of this lib for free.
@ConnorL33t
Did you try to play with KeepAlive
and syncTimeout
settings? Or resolveDns
?
2 weeks so far without those errors on 2.0.519.65453.
Just downgrade a little till the issue is solved.
@ConnorL33t I totally understand that and I do not deny that. @mgravell apologies for some very blunt words. However, my frustration was directed more towards Azure Redis team and their recommendation to go with stackexhange driver initially and then over the time problems we have faced in production environments.
And definitely, thanks are in order for being us use this library.
If it would be a trivial code issue - then everybody would experience it. Millions of StackExchange.Redis NuGet downloads and only a thousand (maybe a hundred) complains. Could it be an issue with server or client environment - Azure Redis (or AWS) or Azure-hosted client? What can we do to help @mgravell to fix this issue?
We found that the issue may be connected to the size of the objects stored in Redis cache. After reducing average object size from ~200Kb to ~20Kb the amount of errors significantly decreased.
Why is the issue closed?
why the issue is close. It's still happenning in my customer's side. The same issue: SocketClosed on flowgear1.redis.cache.windows.net:6380/Interactive, Idle/MarkProcessed, last: SET, origin: ReadFromPipe, outstanding: 0, last-read: 38s ago, last-write: 92s ago, keep-alive: 60s, state: ConnectedEstablished, mgr: 9 of 10 available, in: 0, last-heartbeat: 34s ago, last-mbeat: 2s ago, global: 2s ago, v: 2.0.601.3402
My apologies - I must have accidentally closed this on mobile
We just deployed an update that downgraded to version 2.0.519.65453
(on Azure Cloud Services using Azure Redis) and over the course of about 8 hours so far, these errors have disappeared. Thanks for tip @Styrna, it definitely appears to be an issue introduced in some version between 2.0.519.65453
and 2.0.571
.
Sorry, I should let the community know. We deployed an update that downgraded NuGet to version 2.0.519 (on Azure Service Fabric using Azure Redis) a few weeks ago and we didn't see this error since. Thank you very much @Styrna. I agree with @jglinsek that something was introduced between 2.0.519 and 2.0.571.
I have the same issue for one week now in the production. It s big problem for us. there are lots of Redis exceptions and our website performance getting low.
No connection is available to service this operation: EXISTS https://localhost:44372/api/ContentAreas?localeId=en-us&contentAreaName=benefitsOfRegistration; UnableToConnect (None, 0-read, last-recv: 0) on xxxxxxx.redis.cache.windows.net:6380/Interactive, Flushed, last: ECHO, origin: ResetNonConnected, outstanding: 9, last-read: 5s ago, last-write: 4s ago, unanswered-write: 5s ago, keep-alive: 60s, state: ConnectedEstablishing, mgr: 10 of 10 available, last-heartbeat: never, global: 0s ago, v: 2.0.519.65453; IOCP: (Busy=0,Free=1000,Min=4,Max=1000), WORKER: (Busy=4,Free=8187,Min=4,Max=8191), Local-CPU: n/a
` public static void SetupCache(Container container) { var log = new StringWriter();
//Cache
container.Register<IConnectionMultiplexer>(() =>
{
var cacheConnection = ConfigurationManager.ConnectionStrings["CacheConnection"].ConnectionString;
var connection = ConnectionMultiplexer.Connect(cacheConnection, log);
return connection;
}, Lifestyle.Singleton);
var logOutput = log.ToString();
container.Register<ICacheService, CacheService>();
}`
2.0.519.65453 is working fine for us.
No this is not working for me.
For me personally helped to downgrade library to 1.*
and change number of IOCP as recommendation says (https://github.com/StackExchange/StackExchange.Redis/blob/master/docs/Timeouts.md).
I would start with increasing IOCP, downgrade may not be required in every case.
Recommendation: Given the above information, in 1.* it's recommend to set the minimum configuration value for IOCP and WORKER threads to something larger than the default value. We can't give one-size-fits-all guidance on what this value should be because the right value for one application will be too high/low for another application. This setting can also impact the performance of other parts of complicated applications, so you need to fine-tune this setting to your specific needs. A good starting place is 200 or 300, then test and tweak as needed.
I’m not sure about the rest of the reported, but in my original report the environment had already set those minimums to much higher than default. The error message has a busy value less than half of the minimum
I’m not sure about the rest of the reported, but in my original report the environment had already set those minimums to much higher than default. The error message has a busy value less than half of the minimum
Busy=0,Free=1000,**Min=4**,Max=1000),
You mean to increase this Min value. How can I do that?
@amwayDarshana You need to increase completionPortThreads https://docs.microsoft.com/en-us/dotnet/api/system.threading.threadpool.setminthreads?view=netcore-2.2
In none of the cases we got the error there is any evidence of IOCP thread exhaustion - the busy count is 0 or lower than the minimum, which means new IOCP thread should be available without delay. I see the same in the above posts as well. How's increasing it would yield anything? Or may be there is a logging bug the output is incorrect or at least not aligned with the value at the moment of the problem occurrence?
The problem root cause identified.
If I change the Allow Access Only vis SSL to No. Timeout exception disappeared.
Azure Redis Port 6379 non ssl Port 6380 for SSL
We are also running in this issue and using 2.0.519 (nuget package). So is there any plans and timeline for a fix?
We are experiencing similar errors (.NET471):
StackExchange.Redis.RedisConnectionException: No connection is available to service this operation:
EVAL; SocketClosed (ReadEndOfStream, last-recv: 0) on ....redis.cache.windows.net:6379/Interactive,
Writing/MarkProcessed, last: EVAL, origin: ReadFromPipe, outstanding: 3, last-read: 0s ago,
last-write: 0s ago, keep-alive: 60s, state: ConnectedEstablished, mgr: 9 of 10 available, in: 0,
in-pipe: 0, out-pipe: 0, last-heartbeat: 0s ago, last-mbeat: 0s ago, global: 0s ago, v: 2.0.601.3402;
IOCP: (Busy=0,Free=1000,Min=4,Max=1000),
WORKER: (Busy=21,Free=32746,Min=80,Max=32767), Local-CPU: n/a
It happens only in some of our servers rather rarely. But when it happens then a multiplexer instance is broken forever for all the Redis commands being sent to the multiplexer. The only workaround that works is to recycle App pool.
Additional info: we have two connection multiplexer instances in our application connected to the same Redis server. One instance is working properly, another one is broken at the same time (and same AppDomain).
Update: A socket for Interactive ConnectionType
is not among active Redis clients (result of Redis command CLIENT LIST
) when errors start to occur. At the same time, socket for Subscription ConnectionType
is anong active Redis clients thus the broken multiplexer instance is able to receive and process messages from subscription channels. It seems to me as if ReconnectRetryPolicy
is not applied for some reason.
@amwayDarshana
Looks like my Azure Redis Cache already has those settings, and has had them through my upgrades/downgrades.
Unfortunately, we still experience this issue even with 2.0.519 (nuget package). It happens not as often as before, but it still happens. And 2.0.519 brought a much worse problem, there is an issue with Redis pub/sub. Time of time, our subscribers stop to receive notifications and we have to restart our VMs.
I still have the exception. version: 2.5 (I tryed 2.0.519 same exceptions) `Microsoft.AspNetCore.Session.SessionMiddleware:Error: Error closing the session.
StackExchange.Redis.RedisConnectionException: No connection is available to service this operation: EVAL; UnableToConnect on localhost:6379/Interactive, Initializing/NotStarted, last: NONE, origin: BeginConnectAsync, outstanding: 0, last-read: 2s ago, last-write: 2s ago, keep-alive: 60s, state: Connecting, mgr: 10 of 10 available, last-heartbeat: never, global: 0s ago, v: 2.0.593.37019; IOCP: (Busy=0,Free=1000,Min=2,Max=1000), WORKER: (Busy=1,Free=32766,Min=2,Max=32767), Local-CPU: n/a ---> StackExchange.Redis.RedisConnectionException: UnableToConnect on localhost:6379/Interactive, Initializing/NotStarted, last: NONE, origin: BeginConnectAsync, outstanding: 0, last-read: 2s ago, last-write: 2s ago, keep-alive: 60s, state: Connecting, mgr: 10 of 10 available, last-heartbeat: never, global: 0s ago, v: 2.0.593.37019
at StackExchange.Redis.TaskExtensions.TimeoutAfter(Task task, Int32 timeoutMs) in C:\projects\stackexchange-redis\src\StackExchange.Redis\TaskExtensions.cs:line 49
at StackExchange.Redis.ConnectionMultiplexer.WaitAllIgnoreErrorsAsync(Task[] tasks, Int32 timeoutMilliseconds, TextWriter log, String caller, Int32 callerLineNumber) in C:\projects\stackexchange-redis\src\StackExchange.Redis\ConnectionMultiplexer.cs:line 705
--- End of inner exception stack trace ---
at StackExchange.Redis.ConnectionMultiplexer.ThrowFailed[T](TaskCompletionSource1 source, Exception unthrownException) in C:\projects\stackexchange-redis\src\StackExchange.Redis\ConnectionMultiplexer.cs:line 2174 --- End of stack trace from previous location where exception was thrown --- at Microsoft.Extensions.Caching.StackExchangeRedis.RedisExtensions.HashMemberGetAsync(IDatabase cache, String key, String[] members) at Microsoft.Extensions.Caching.StackExchangeRedis.RedisCache.GetAndRefreshAsync(String key, Boolean getData, CancellationToken token) at Microsoft.Extensions.Caching.StackExchangeRedis.RedisCache.RefreshAsync(String key, CancellationToken token) at Microsoft.AspNetCore.Session.DistributedSession.CommitAsync(CancellationToken cancellationToken) at Microsoft.AspNetCore.Session.SessionMiddleware.Invoke(HttpContext context)
Is there any alternative to solve the problem?
Hi,
I'm facing the same problem with Nuget version 2.0.519.
"No connection is available to service this operation: GET xxxx; UnableToConnect on tnazeu-pd-mhs-rediscache.redis.cache.windows.net:6380/Interactive, Initializing, last: NONE, origin: BeginConnectAsync, outstanding: 0, last-read: 0s ago, last-write: 0s ago, unanswered-write: 385712s ago, keep-alive: 60s, state: Connecting, mgr: 10 of 10 available, last-heartbeat: never, global: 0s ago, v: 2.0.519.65453; IOCP: (Busy=3,Free=997,Min=24,Max=1000), WORKER: (Busy=4,Free=32763,Min=24,Max=32767), Local-CPU: n/a"
This is affecting production, is there any update/alternative for this bug yet?
Thanks, Aiyappa
Make sure in local machine have SSL and TSL communication enabled. If you are using virtual PC it may disabled by the network administrator.
Try to user wireshark network trace tool. And capture a trace calls from your application to server. and check which ports are actually using. You can track down the actual problem by examining Ethernet port requests.
I solved my Redis no connection issue after checking network trace. it was our side issue. We pointed our wrong server in wrong environment.
@amwayDarshana, your problem is related to inability to connect to Redis due to mismatch between how the Redis instance was configured (to deny non-SSL connections) and how the client was setup. The original thread is about having Redis client which had been working for minutes, hours even days fine that suddenly breaks and begins to throw RedisConnectionException for no apparent reason - the Redis cache instance is available, up and running and nothing had been changed on either the client nor the server.
same issue here w/ .601
Same issue here StackExchange.Redis.RedisConnectionException: SocketClosed on xxxxx
dotnet core 2.2 project running on Ubuntu 18.04 same issue both v1.2.6 and v2.0.601
I suggest you to contact Microsoft support . They helped me to solve the problem. I did not update stackexchange.version buy they check network traffic using wireshark and solved the issue.
I highly suggest you to check network traffic log. Check what are the requests going though Ethernet port. Using wireshark. Check TSL request and how SSL handshake works.
Use psping and check any firewall issues.
Please let me know any progress after you done.
Make sure get master database may be helpful (?)
IDatabase GetDatabase(ConnectionMultiplexer conn)
{
return (from endPoint in conn.GetEndPoints()
select conn.GetServer(endPoint) into server
where !server.IsSlave
select server.Multiplexer.GetDatabase())
.FirstOrDefault();
}
@mgravell Any news on this?
If it matters, we had also reverted back to 2.0.519.65453. Now almost three weeks, no issue of that sort. With our mixture of Redis requests no noticeable performance impact as well.
Any update?
I have tried downgrading to 2.0.519 from 2.0.601 but I am still get the errors.
Hi.
Recently updated our server with StackExchange.Redis v2.0.600. About a day or two into running the library I started receiving thousands of these errors
RedisConnectionException: No connection is available to service this operation; It was not possible to connect to the redis server(s). To create a disconnected multiplexer, disable AbortOnConnectFail. ConnectTimeout; IOCP: (Busy=45,Free=955,Min=1000,Max=1000), WORKER: (Busy=43,Free=32724,Min=100,Max=32767), Local-CPU: n/a
Immediate issues were resolved by a web app restart, shortly after I reverted back to an older version of StackExchange.Redis.
Running Azure Redis Cache, ~20 client connections during these errors, ~120KiB throughput. 100Mb cache size. Redis Resource health shows "good"