Issue with Redis cache in Windows.

[olsizotaj]

Hi,

We already have installed redis cache (Master slave - sentineil) on windows machine (Virtual Machine). From web server (another IIS Vistual machine. Bandwidth between two server is 1 GBps) we get data from redis cache. The data are stored as HASH dataset in cache. When we generate 600 - 1000 concurrent requests only to get data from redis an error is thrown by redis in its log file. When this issue happens redis consumes all the RAM memory (around 24 GB) and the windows stops working. The size of our data that are stored in RAM are less than 5 GB. The population of this redis (slave node) is made by its master. And the master redis population is provided by another windows service. All requests are pointed to slave node and the situation of master node is okay.

For all requests from our web app we created only one connection for all requests like below: StackExchange.Redis.ConnectionMultiplexer redis; RedisConnect.Hosts = "ip of slave host:port"; redis = RedisConnect.GetNoticeConnection; NewtonsoftSerializer serializer = new NewtonsoftSerializer(); client = new StackExchangeRedisCacheClient(redis, serializer);

Redis error in its lo file: === REDIS BUG REPORT START: Cut & paste starting from here === Redis version: 3.2.100 [4652] 09 Feb 14:40:53.530 # --- EXCEPTION_ACCESS_VIOLATION [4652] 09 Feb 14:40:53.545 # --- STACK TRACE redis-server.exe!LogStackTrace(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:95)(0x0013E2C0, 0x0013FF90, 0x00000001, 0x4013A7F8) redis-server.exe!UnhandledExceptiontHandler(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00000000, 0x00000000, 0x00000000, 0x0013E268) KERNELBASE.dll!UnhandledExceptionFilter(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00000000, 0x198C1DA0, 0x00000000, 0x0013FF20) ntdll.dll!memset(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x0013EEF0, 0x13F0E37E, 0x40140E48, 0x00000000) ntdll.dll!_C_specific_handler(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00000000, 0x0013E430, 0x0013EEF0, 0x0013EEF0) ntdll.dll!_chkstk(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00000001, 0x1C410000, 0x00000000, 0x1C54409C) ntdll.dll!RtlRaiseException(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00223B30, 0x02E55535, 0x00867DAF, 0x02AE7788) ntdll.dll!KiUserExceptionDispatcher(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0xBC52CE96, 0x400951B9, 0x02807070, 0x02807070) redis-server.exe!rdbSaveStringObject(c:\release\redis\src\rdb.c:391)(0x02807070, 0x00867D9D, 0xBC52CF80, 0x0236F220) redis-server.exe!rdbSaveObject(c:\release\redis\src\rdb.c:678)(0x00000002, 0x0013F2E0, 0x0013F2E0, 0x00000001) redis-server.exe!rdbSaveKeyValuePair(c:\release\redis\src\rdb.c:721)(0x0013F2E0, 0x00000001, 0x0013F2E0, 0x02807040) redis-server.exe!rdbSaveRio(c:\release\redis\src\rdb.c:814)(0x40167210, 0x00720000, 0x00000005, 0x00B22754) redis-server.exe!rdbSave(c:\release\redis\src\rdb.c:884)(0x00720000, 0x00720000, 0x589560D4, 0x00000005) redis-server.exe!QForkChildInit(c:\release\redis\src\win32_interop\win32_qfork.cpp:337)(0x00000005, 0x00000000, 0x00226200, 0x00000005) redis-server.exe!QForkStartup(c:\release\redis\src\win32_interop\win32_qfork.cpp:515)(0x00000006, 0x00000000, 0xFFFFB000, 0x00226EA0) redis-server.exe!main(c:\release\redis\src\win32_interop\win32_qfork.cpp:1240)(0x00000000, 0x00000000, 0x00000000, 0xFFFFB000) redis-server.exe!__tmainCRTStartup(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x400459C0, 0x00000000, 0x00000000) KERNEL32.DLL!BaseThreadInitThunk(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x1A8C13B0, 0x00000000, 0x00000000, 0x00000000) ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000) ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000)

The error in StackExchange.Redis library: 2017-02-09 11:46:30,632 [P5088/D2/T118] ERROR APP_Portal Search 0 - ID: 91aa3441-5b04-4b95-879c-f024ecc4c6b1 System.TimeoutException: Timeout performing HVALS APPPublicServices.Model.TenderNotice, inst: 1, queue: 1724, qu: 0, qs: 1724, qc: 0, wr: 0, wq: 0, in: 0, ar: 0, clientName: Notice Repository, IOCP: (Busy=0,Free=1000,Min=8,Max=1000), WORKER: (Busy=18,Free=32749,Min=100,Max=32767), Local-CPU: unavailable (Please take a look at this article for some common client-side issues that can cause timeouts: https://github.com/StackExchange/StackExchange.Redis/tree/master/Docs/Timeouts.md) at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor1 processor, ServerEndPoint server) at StackExchange.Redis.RedisBase.ExecuteSync[T](Message message, ResultProcessor1 processor, ServerEndPoint server) at StackExchange.Redis.RedisDatabase.HashValues(RedisKey key, CommandFlags flags) at StackExchange.Redis.Extensions.Core.StackExchangeRedisCacheClient.HashValues[T](String hashKey, CommandFlags commandFlags) at APPPublicServices.Infrastructure.Services.CacheResourceRepository2.Finby(Expression`1 predicate) at APPPublicServices.Domain.Services.TenderNoticeService.GetNotices(NoticeFilter filter, Int32 pageIndex, Int32 pageSize) at APPPublicPortal.Controllers.SearchController.Search(RenderModel model)

Please help us to solve this issue. Thank you