search

microsoftarchive / redis

Redis is an in-memory database that persists on disk. The data model is key-value, but many different kind of values are supported: Strings, Lists, Sets, Sorted Sets, Hashes
http://redis.io
Other
20.82k stars 5.37k forks source link

Bug report on Redis 3.2.100 #542

Open zoirmi opened 7 years ago

zoirmi commented 7 years ago

=== REDIS BUG REPORT START: Cut & paste starting from here === Redis version: 3.2.100 [8716] 04 Apr 10:39:01.639 # --- EXCEPTION_ACCESS_VIOLATION [8716] 04 Apr 10:39:01.641 # --- STACK TRACE redis-server.exe!LogStackTrace(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:95)(0x0013E2C0, 0x0013FF90, 0x00000001, 0x4013A7F8) redis-server.exe!UnhandledExceptiontHandler(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00000000, 0x00000000, 0x00000000, 0x0199E8BB) KERNELBASE.dll!UnhandledExceptionFilter(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00000003, 0x21AC1AD0, 0x00000000, 0x245B8736) ntdll.dll!memset(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x0013EEF0, 0x02EC8E04, 0x40140E48, 0x00000000) ntdll.dll!_C_specific_handler(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00000000, 0x0013E430, 0x0013EEF0, 0x0013EEF0) ntdll.dll!_chkstk(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00000001, 0x24580000, 0x00000000, 0x246B6204) ntdll.dll!RtlRaiseException(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x019A6396, 0x019A0A51, 0x00000007, 0x023C3677) ntdll.dll!KiUserExceptionDispatcher(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0xA0BA364D, 0x400951B9, 0x02C07070, 0x02C07070) redis-server.exe!rdbSaveStringObject(c:\release\redis\src\rdb.c:391)(0x02C07070, 0x00000001, 0xA0BA3702, 0x0297C140) redis-server.exe!rdbSaveObject(c:\release\redis\src\rdb.c:678)(0x38703293, 0x0013F2E0, 0x0013F2E0, 0x00000001) redis-server.exe!rdbSaveKeyValuePair(c:\release\redis\src\rdb.c:721)(0x0013F2E0, 0x00000001, 0x0013F2E0, 0x02C070FC) redis-server.exe!rdbSaveRio(c:\release\redis\src\rdb.c:814)(0x40167210, 0x00830000, 0x00000005, 0x00C32754) redis-server.exe!rdbSave(c:\release\redis\src\rdb.c:884)(0x00830000, 0x00830000, 0x58EE0211, 0x00000005) redis-server.exe!QForkChildInit(c:\release\redis\src\win32_interop\win32_qfork.cpp:337)(0x00000005, 0x00000000, 0x002570D0, 0x00000005) redis-server.exe!QForkStartup(c:\release\redis\src\win32_interop\win32_qfork.cpp:515)(0x00000006, 0x00000000, 0xFFFF6000, 0x00258480) redis-server.exe!main(c:\release\redis\src\win32_interop\win32_qfork.cpp:1240)(0x00000000, 0x00000000, 0x00000000, 0xFFFF6000) redis-server.exe!__tmainCRTStartup(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x400459C0, 0x00000000, 0x00000000) KERNEL32.DLL!BaseThreadInitThunk(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x222F13B0, 0x00000000, 0x00000000, 0x00000000) ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000) ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000) [8716] 04 Apr 10:39:01.679 # --- INFO OUTPUT

Server

redis_version:3.2.100 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:dd26f1f93c5130ee redis_mode:standalone os:Windows
arch_bits:64 multiplexing_api:WinSock_IOCP process_id:8716 run_id:d2cabf9d9932af2e65ae9dbd73b84bab5841ac64 tcp_port:6379 uptime_in_seconds:166669 uptime_in_days:1 hz:10 lru_clock:14900133 executable:R:\redis-server.exe config_file:R:\redis.windows.conf

Clients

connected_clients:1466 client_longest_output_list:0 client_biggest_input_buf:0 blocked_clients:0

Memory

used_memory:992 used_memory_human:992B used_memory_rss:124319952 used_memory_rss_human:118.56M used_memory_peak:166022592 used_memory_peak_human:158.33M total_system_memory:0 total_system_memory_human:0B used_memory_lua:0 used_memory_lua_human:0B maxmemory:8589934592 maxmemory_human:8.00G maxmemory_policy:noeviction mem_fragmentation_ratio:125322.53 mem_allocator:jemalloc-3.6.0

Persistence

loading:0 rdb_changes_since_last_save:13853 rdb_bgsave_in_progress:1 rdb_last_save_time:1491295080 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:0 rdb_current_bgsave_time_sec:1491295142 aof_enabled:0 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_last_write_status:ok

Stats

total_connections_received:536613 total_commands_processed:368736875 instantaneous_ops_per_sec:1082 total_net_input_bytes:154832354122 total_net_output_bytes:2891073673938 instantaneous_input_kbps:978.30 instantaneous_output_kbps:16108.52 rejected_connections:0 sync_full:0 sync_partial_ok:0 sync_partial_err:0 expired_keys:144954 evicted_keys:0 keyspace_hits:252904781 keyspace_misses:75458441 pubsub_channels:116 pubsub_patterns:0 latest_fork_usec:7977 migrate_cached_sockets:0

Replication

role:master connected_slaves:0 master_repl_offset:0 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0

CPU

used_cpu_sys:0.08 used_cpu_user:0.47 used_cpu_sys_children:0.00 used_cpu_user_children:0.00

Commandstats

Cluster

cluster_enabled:0

Keyspace

db0:keys=22189,expires=268,avg_ttl=42368 [8716] 04 Apr 10:39:01.699 # === REDIS BUG REPORT END. Make sure to include from START to END. ===

granstel commented 7 years ago

I have the same issue

AbhijeetVR commented 7 years ago

We are also facing same issue. Initially we were getting a Windows message popup saying - "Redis server has stooped working". For such instances, found the following error in redis log file.

Redis can not continue. Exiting.
[5496] 15 Apr 06:25:58.340 # fork operation failed
[5496] 15 Apr 06:25:58.464 # Background saving terminated by signal 1
[5496] 15 Apr 06:26:04.094 * 1 changes in 300 seconds. Saving...
[5496] 15 Apr 06:26:04.216 * Background saving started by pid 9736
[9736] 15 Apr 00:56:04.275 # 
The Windows version of Redis reserves heap memory from the system paging file
for sharing with the forked process used for persistence operations.At this time there is insufficient contiguous free space available in the
system paging file. You may increase the size of the system paging file.
Sometimes a reboot will defragment the system paging file sufficiently for
this operation to complete successfully.

To resolve, required virtual memory (page file) setting was made and it helped.

But, for few other Redis server instances running on separate nodes, where we were getting same windows message as "Redis server has stooped working", this resolution didn't work and for those instances we found the error mentioned in first post. If anyone has observed the same issue and found any resolution on this, please do update as to what steps can be adopted to resolve the same.

C421FD commented 7 years ago

I have the same problem with using SignalR + Redis in backplane. I use w2008r2, page file size = 400mb, snapshotting is enabled. SignalR don't store values to redis, SignalR using only pub/sub feature. What the happen????

=== REDIS BUG REPORT START: Cut & paste starting from here === [12084] 05 Sep 18:59:50.148 # Redis version: 3.0.504 [12084] 05 Sep 18:59:50.148 # --- EXCEPTION_ACCESS_VIOLATION [12084] 05 Sep 18:59:50.148 # --- STACK TRACE redis-server.exe!LogStackTrace(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:95)(0x0012E670, 0x0012FF90, 0x00000001, 0x40124730) redis-server.exe!UnhandledExceptiontHandler(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00000001, 0x00000000, 0x00000001, 0x7727E068) kernel32.dll!UnhandledExceptionFilter(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x0012E670, 0x00000006, 0x00000000, 0x00000001) ntdll.dll!EtwEventSetInformation(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x0012F2B0, 0x00000278, 0x4012AC28, 0x00000000) ntdll.dll!_C_specific_handler(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00130000, 0x0012FF90, 0x0012FF90, 0x7736F818) ntdll.dll!RtlDecodePointer(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00130000, 0x7721DD88, 0x000127EC, 0x00000000) ntdll.dll!RtlUnwindEx(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x0012F2B0, 0x0012EDC0, 0x00000000, 0x00000000) ntdll.dll!KiUserExceptionDispatcher(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x4002C9BD, 0x00000000, 0x00000001, 0x00000000) redis-server.exe!write(c:\release\redis\src\win32_interop\win32_stacktrace.cpp:185)(0x00000000, 0x00000001, 0x00000000, 0x00000030) redis-server.exe!je_arenas_extend(c:\release\redis\deps\jemalloc-win\src\jemalloc.c:166)(0x00000001, 0x00000000, 0x00000030, 0x00001000) redis-server.exe!malloc_init_hard(c:\release\redis\deps\jemalloc-win\src\jemalloc.c:768)(0x00000000, 0x00000030, 0x00000001, 0x00000009) redis-server.exe!je_malloc(c:\release\redis\deps\jemalloc-win\src\jemalloc.c:956)(0x0012F6E0, 0x00000030, 0x00000009, 0x40150210) redis-server.exe!zmalloc(c:\release\redis\src\zmalloc.c:142)(0x0012F6E0, 0x0012F6E0, 0x00000009, 0x00000006) redis-server.exe!rdbSaveRio(c:\release\redis\src\rdb.c:683)(0x40150210, 0x00C20000, 0x00000005, 0x01022754) redis-server.exe!rdbSave(c:\release\redis\src\rdb.c:758)(0x00C20000, 0x59A211CA, 0x00C20000, 0x00000005) redis-server.exe!QForkChildInit(c:\release\redis\src\rdb.c:758)(0x00000005, 0x00000000, 0x001A40B0, 0x00000005) redis-server.exe!QForkStartup(c:\release\redis\src\rdb.c:758)(0x00000006, 0x00000000, 0x00000000, 0x001A2CB0) redis-server.exe!main(c:\release\redis\src\rdb.c:758)(0x00000000, 0x00000000, 0x00000000, 0x00000000) redis-server.exe!__tmainCRTStartup(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000) kernel32.dll!BaseThreadInitThunk(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000) ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000) ntdll.dll!RtlUserThreadStart(f:\dd\vctools\crt\crtw32\startup\crt0.c:255)(0x00000000, 0x00000000, 0x00000000, 0x00000000) [12084] 05 Sep 18:59:50.148 # --- INFO OUTPUT [3424] 05 Sep 18:59:50.242 # fork operation failed [3424] 05 Sep 18:59:50.242 # Background saving terminated by signal 1 [3424] 05 Sep 18:59:56.023 10 changes in 300 seconds. Saving... [3424] 05 Sep 18:59:56.023 Background saving started by pid 368 [368] 05 Sep 18:59:56.102 # The Windows version of Redis reserves heap memory from the system paging file for sharing with the forked process used for persistence operations.At this time there is insufficient contiguous free space available in the system paging file. You may increase the size of the system paging file. Sometimes a reboot will defragment the system paging file sufficiently for this operation to complete successfully.

Redis can not continue. Exiting. [3424] 05 Sep 18:59:56.133 # fork operation failed

7Eric9 commented 6 years ago

same issue on my pc

suwenkun1126 commented 6 years ago

I have the same issue,any solution?

Theboss83 commented 6 years ago

I have the same issue on my pc. Any solution? Is this Thread managed? It's possible that since 7th of April no one found a solution for that? Should we wait for release 4.0?

tporadowski commented 6 years ago

Since this fork is no longer maintained - please try the updated port of Redis 4.0.2 from tporadowski/redis and its alpha release. If this issue still persists - please report it there with as many details as possible.