search

aviggiano / redis-roaring

Roaring Bitmaps for Redis
MIT License
345 stars 55 forks source link

RedisRoaring crashing on Arm64 Docker (Apple M1) #101

Closed BenjiLewis closed 2 years ago

BenjiLewis commented 2 years ago

Redis Roaring is not working on MacBook M1 . There is a known issue with docker images that don't support the ARM architecture that can be seem here. The issue is now closed as there are now new docker images that support the architecture (as per this comment)

Would it be possible to add a new Docker Image to support the new architecture?

compose-redis_roaring-1         | === REDIS BUG REPORT START: Cut & paste starting from here ===
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.879 # Redis 6.2.4 crashed by signal: 11, si_code: 1
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.879 # Accessing address: 0x20200ff00fe
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.879 # Crashed running the instruction at: 0x4000047c3b
compose-redis_roaring-1         |
compose-redis_roaring-1         | ------ STACK TRACE ------
compose-redis_roaring-1         | EIP:
compose-redis_roaring-1         | /usr/local/bin/redis-server *:6379(dictAddRaw+0x16b)[0x4000047c3b]
compose-redis_roaring-1         |
compose-redis_roaring-1         | Backtrace:
compose-redis_roaring-1         | /lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x400217e730]
compose-redis_roaring-1         | /usr/local/bin/redis-server *:6379(dictAddRaw+0x16b)[0x4000047c3b]
compose-redis_roaring-1         | /usr/local/bin/redis-server *:6379(dictAdd+0x11)[0x4000047dd1]
compose-redis_roaring-1         | /usr/local/bin/redis-server *:6379(dbAddRDBLoad+0x13)[0x400006f0f3]
compose-redis_roaring-1         | /usr/local/bin/redis-server *:6379(rdbLoadRio+0x2a6)[0x400007d866]
compose-redis_roaring-1         | /usr/local/bin/redis-server *:6379(rdbLoad+0x51)[0x400007e311]
compose-redis_roaring-1         | /usr/local/bin/redis-server *:6379(readSyncBulkPayload+0x142)[0x4000073a82]
compose-redis_roaring-1         | /usr/local/bin/redis-server *:6379(+0xf3e88)[0x40000f3e88]
compose-redis_roaring-1         | /usr/local/bin/redis-server *:6379(aeProcessEvents+0x2a1)[0x4000045ce1]
compose-redis_roaring-1         | /usr/local/bin/redis-server *:6379(aeMain+0x1d)[0x4000045f3d]
compose-redis_roaring-1         | /usr/local/bin/redis-server *:6379(main+0x314)[0x40000426f4]
compose-redis_roaring-1         | /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb)[0x40021b109b]
compose-redis_roaring-1         | /usr/local/bin/redis-server *:6379(_start+0x2a)[0x4000042bda]
compose-redis_roaring-1         |
compose-redis_roaring-1         | ------ REGISTERS ------
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 #
compose-redis_roaring-1         | RAX:0000000000000000 RBX:0000004005afae83
compose-redis_roaring-1         | RCX:0000000000000042 RDX:00000040029d0350
compose-redis_roaring-1         | RDI:0000004005afae83 RSI:00000040001893ac
compose-redis_roaring-1         | RBP:add208bc77afff87 RSP:0000004001c2a370
compose-redis_roaring-1         | R8 :0000000000000000 R9 :000000400295f430
compose-redis_roaring-1         | R10:0000004005afaec3 R11:0000000000000002
compose-redis_roaring-1         | R12:0000020200ff00fe R13:0000000000000000
compose-redis_roaring-1         | R14:000000400295f420 R15:000000000003ff87
compose-redis_roaring-1         | RIP:0000004000047c3b EFL:0000000000000202
compose-redis_roaring-1         | CSGSFS:002b000000000033
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a37f) -> 000000400006f0f3
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a37e) -> 0000004001c2a8f0
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a37d) -> 0000004005afae83
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a37c) -> 0000000000000000
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a37b) -> 0000004000047dd1
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a37a) -> 0000004001c2a480
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a379) -> 000000400c3d7b40
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a378) -> 0000004005afae83
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a377) -> 000000400c3d7b40
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a376) -> 0000004005afae83
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a375) -> 000000400295f420
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a374) -> 7ffffffe00000000
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a373) -> 000000400295f430
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a372) -> 000000400295f470
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a371) -> 00000000001ffc38
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.887 # (0000004001c2a370) -> 000000400295f430
compose-redis_roaring-1         |
compose-redis_roaring-1         | ------ INFO OUTPUT ------
compose-redis_roaring-1         | # Server
compose-redis_roaring-1         | redis_version:6.2.4
compose-redis_roaring-1         | redis_git_sha1:00000000
compose-redis_roaring-1         | redis_git_dirty:0
compose-redis_roaring-1         | redis_build_id:49ac482504d79440
compose-redis_roaring-1         | redis_mode:standalone
compose-redis_roaring-1         | os:Linux 5.10.76-linuxkit x86_64
compose-redis_roaring-1         | arch_bits:64
compose-redis_roaring-1         | multiplexing_api:epoll
compose-redis_roaring-1         | atomicvar_api:c11-builtin
compose-redis_roaring-1         | gcc_version:8.3.0
compose-redis_roaring-1         | process_id:1
compose-redis_roaring-1         | process_supervised:no
compose-redis_roaring-1         | run_id:71af10c34bc5f54e9b8141479da1b4a8d2e3d23f
compose-redis_roaring-1         | tcp_port:6379
compose-redis_roaring-1         | server_time_usec:1650620219818320
compose-redis_roaring-1         | uptime_in_seconds:42
compose-redis_roaring-1         | uptime_in_days:0
compose-redis_roaring-1         | hz:10
compose-redis_roaring-1         | configured_hz:10
compose-redis_roaring-1         | lru_clock:6453050
compose-redis_roaring-1         | executable:/usr/local/bin/redis-server
compose-redis_roaring-1         | config_file:
compose-redis_roaring-1         | io_threads_active:0
compose-redis_roaring-1         |
compose-redis_roaring-1         | # Clients
compose-redis_roaring-1         | connected_clients:0
compose-redis_roaring-1         | cluster_connections:0
compose-redis_roaring-1         | maxclients:10000
compose-redis_roaring-1         | client_recent_max_input_buffer:0
compose-redis_roaring-1         | client_recent_max_output_buffer:0
compose-redis_roaring-1         | blocked_clients:0
compose-redis_roaring-1         | tracking_clients:0
compose-redis_roaring-1         | clients_in_timeout_table:0
compose-redis_roaring-1         |
compose-redis_roaring-1         | # Memory
compose-redis_roaring-1         | used_memory:4364912
compose-redis_roaring-1         | used_memory_human:4.16M
compose-redis_roaring-1         | used_memory_rss:0
compose-redis_roaring-1         | used_memory_rss_human:0B
compose-redis_roaring-1         | used_memory_peak:256380728
compose-redis_roaring-1         | used_memory_peak_human:244.50M
compose-redis_roaring-1         | used_memory_peak_perc:1.70%
compose-redis_roaring-1         | used_memory_overhead:2969776
compose-redis_roaring-1         | used_memory_startup:821232
compose-redis_roaring-1         | used_memory_dataset:1395136
compose-redis_roaring-1         | used_memory_dataset_perc:39.37%
compose-redis_roaring-1         | allocator_allocated:256837648
compose-redis_roaring-1         | allocator_active:257142784
compose-redis_roaring-1         | allocator_resident:262643712
compose-redis_roaring-1         | total_system_memory:27287318528
compose-redis_roaring-1         | total_system_memory_human:25.41G
compose-redis_roaring-1         | used_memory_lua:37888
compose-redis_roaring-1         | used_memory_lua_human:37.00K
compose-redis_roaring-1         | used_memory_scripts:0
compose-redis_roaring-1         | used_memory_scripts_human:0B
compose-redis_roaring-1         | number_of_cached_scripts:0
compose-redis_roaring-1         | maxmemory:0
compose-redis_roaring-1         | maxmemory_human:0B
compose-redis_roaring-1         | maxmemory_policy:noeviction
compose-redis_roaring-1         | allocator_frag_ratio:1.00
compose-redis_roaring-1         | allocator_frag_bytes:305136
compose-redis_roaring-1         | allocator_rss_ratio:1.02
compose-redis_roaring-1         | allocator_rss_bytes:5500928
compose-redis_roaring-1         | rss_overhead_ratio:0.00
compose-redis_roaring-1         | rss_overhead_bytes:-262643712
compose-redis_roaring-1         | mem_fragmentation_ratio:0.00
compose-redis_roaring-1         | mem_fragmentation_bytes:-256359920
compose-redis_roaring-1         | mem_not_counted_for_evict:0
compose-redis_roaring-1         | mem_replication_backlog:0
compose-redis_roaring-1         | mem_clients_slaves:0
compose-redis_roaring-1         | mem_clients_normal:0
compose-redis_roaring-1         | mem_aof_buffer:0
compose-redis_roaring-1         | mem_allocator:jemalloc-5.1.0
compose-redis_roaring-1         | active_defrag_running:0
compose-redis_roaring-1         | lazyfree_pending_objects:0
compose-redis_roaring-1         | lazyfreed_objects:0
compose-redis_roaring-1         |
compose-redis_roaring-1         | # Persistence
compose-redis_roaring-1         | loading:1
compose-redis_roaring-1         | current_cow_size:0
compose-redis_roaring-1         | current_cow_size_age:0
compose-redis_roaring-1         | current_fork_perc:0.00
compose-redis_roaring-1         | current_save_keys_processed:0
compose-redis_roaring-1         | current_save_keys_total:0
compose-redis_roaring-1         | rdb_changes_since_last_save:0
compose-redis_roaring-1         | rdb_bgsave_in_progress:0
compose-redis_roaring-1         | rdb_last_save_time:1650620177
compose-redis_roaring-1         | rdb_last_bgsave_status:ok
compose-redis_roaring-1         | rdb_last_bgsave_time_sec:-1
compose-redis_roaring-1         | rdb_current_bgsave_time_sec:-1
compose-redis_roaring-1         | rdb_last_cow_size:0
compose-redis_roaring-1         | aof_enabled:0
compose-redis_roaring-1         | aof_rewrite_in_progress:0
compose-redis_roaring-1         | aof_rewrite_scheduled:0
compose-redis_roaring-1         | aof_last_rewrite_time_sec:-1
compose-redis_roaring-1         | aof_current_rewrite_time_sec:-1
compose-redis_roaring-1         | aof_last_bgrewrite_status:ok
compose-redis_roaring-1         | aof_last_write_status:ok
compose-redis_roaring-1         | aof_last_cow_size:0
compose-redis_roaring-1         | module_fork_in_progress:0
compose-redis_roaring-1         | module_fork_last_cow_size:0
compose-redis_roaring-1         | loading_start_time:1650620219
compose-redis_roaring-1         | loading_total_bytes:214341529
compose-redis_roaring-1         | loading_rdb_used_mem:283646856
compose-redis_roaring-1         | loading_loaded_bytes:0
compose-redis_roaring-1         | loading_loaded_perc:0.00
compose-redis_roaring-1         | loading_eta_seconds:1
compose-redis_roaring-1         |
compose-redis_roaring-1         | # Stats
compose-redis_roaring-1         | total_connections_received:0
compose-redis_roaring-1         | total_commands_processed:0
compose-redis_roaring-1         | instantaneous_ops_per_sec:0
compose-redis_roaring-1         | total_net_input_bytes:214341529
compose-redis_roaring-1         | total_net_output_bytes:0
compose-redis_roaring-1         | instantaneous_input_kbps:6776.11
compose-redis_roaring-1         | instantaneous_output_kbps:0.00
compose-redis_roaring-1         | rejected_connections:0
compose-redis_roaring-1         | sync_full:0
compose-redis_roaring-1         | sync_partial_ok:0
compose-redis_roaring-1         | sync_partial_err:0
compose-redis_roaring-1         | expired_keys:0
compose-redis_roaring-1         | expired_stale_perc:0.00
compose-redis_roaring-1         | expired_time_cap_reached_count:0
compose-redis_roaring-1         | expire_cycle_cpu_milliseconds:0
compose-redis_roaring-1         | evicted_keys:0
compose-redis_roaring-1         | keyspace_hits:0
compose-redis_roaring-1         | keyspace_misses:0
compose-redis_roaring-1         | pubsub_channels:0
compose-redis_roaring-1         | pubsub_patterns:0
compose-redis_roaring-1         | latest_fork_usec:0
compose-redis_roaring-1         | total_forks:0
compose-redis_roaring-1         | migrate_cached_sockets:0
compose-redis_roaring-1         | slave_expires_tracked_keys:0
compose-redis_roaring-1         | active_defrag_hits:0
compose-redis_roaring-1         | active_defrag_misses:0
compose-redis_roaring-1         | active_defrag_key_hits:0
compose-redis_roaring-1         | active_defrag_key_misses:0
compose-redis_roaring-1         | tracking_total_keys:0
compose-redis_roaring-1         | tracking_total_items:0
compose-redis_roaring-1         | tracking_total_prefixes:0
compose-redis_roaring-1         | unexpected_error_replies:0
compose-redis_roaring-1         | total_error_replies:0
compose-redis_roaring-1         | dump_payload_sanitizations:0
compose-redis_roaring-1         | total_reads_processed:0
compose-redis_roaring-1         | total_writes_processed:0
compose-redis_roaring-1         | io_threaded_reads_processed:0
compose-redis_roaring-1         | io_threaded_writes_processed:0
compose-redis_roaring-1         |
compose-redis_roaring-1         | # Replication
compose-redis_roaring-1         | role:slave
compose-redis_roaring-1         | master_host:10.5.0.2
compose-redis_roaring-1         | master_port:9001
compose-redis_roaring-1         | master_link_status:down
compose-redis_roaring-1         | master_last_io_seconds_ago:-1
compose-redis_roaring-1         | master_sync_in_progress:1
compose-redis_roaring-1         | slave_repl_offset:1
compose-redis_roaring-1         | master_sync_total_bytes:214341529
compose-redis_roaring-1         | master_sync_read_bytes:214341529
compose-redis_roaring-1         | master_sync_left_bytes:0
compose-redis_roaring-1         | master_sync_perc:100.00
compose-redis_roaring-1         | master_sync_last_io_seconds_ago:1
compose-redis_roaring-1         | master_link_down_since_seconds:-1
compose-redis_roaring-1         | slave_priority:100
compose-redis_roaring-1         | slave_read_only:0
compose-redis_roaring-1         | replica_announced:1
compose-redis_roaring-1         | connected_slaves:0
compose-redis_roaring-1         | master_failover_state:no-failover
compose-redis_roaring-1         | master_replid:739a95e73aa0248df36918072e3c92f315b45d61
compose-redis_roaring-1         | master_replid2:0000000000000000000000000000000000000000
compose-redis_roaring-1         | master_repl_offset:13617880758
compose-redis_roaring-1         | second_repl_offset:-1
compose-redis_roaring-1         | repl_backlog_active:0
compose-redis_roaring-1         | repl_backlog_size:1048576
compose-redis_roaring-1         | repl_backlog_first_byte_offset:0
compose-redis_roaring-1         | repl_backlog_histlen:0
compose-redis_roaring-1         |
compose-redis_roaring-1         | # CPU
compose-redis_roaring-1         | used_cpu_sys:3.746705
compose-redis_roaring-1         | used_cpu_user:5.691536
compose-redis_roaring-1         | used_cpu_sys_children:0.025411
compose-redis_roaring-1         | used_cpu_user_children:0.072327
compose-redis_roaring-1         | used_cpu_sys_main_thread:3.734287
compose-redis_roaring-1         | used_cpu_user_main_thread:5.645092
compose-redis_roaring-1         |
compose-redis_roaring-1         | # Modules
compose-redis_roaring-1         | module:name=REDIS-ROARING,ver=1,api=1,filters=0,usedby=[],using=[],options=[]
compose-redis_roaring-1         |
compose-redis_roaring-1         | # Commandstats
compose-redis_roaring-1         |
compose-redis_roaring-1         | # Errorstats
compose-redis_roaring-1         |
compose-redis_roaring-1         | # Cluster
compose-redis_roaring-1         | cluster_enabled:0
compose-redis_roaring-1         |
compose-redis_roaring-1         | # Keyspace
compose-redis_roaring-1         | db0:keys=1284,expires=0,avg_ttl=0
compose-redis_roaring-1         |
compose-redis_roaring-1         | ------ CLIENT LIST OUTPUT ------
compose-redis_roaring-1         |
compose-redis_roaring-1         | ------ MODULES INFO OUTPUT ------
compose-redis_roaring-1         |
compose-redis_roaring-1         | ------ FAST MEMORY TEST ------
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.893 # Bio thread for job type #0 terminated
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.894 # Bio thread for job type #1 terminated
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:36:59.894 # Bio thread for job type #2 terminated
compose-redis_roaring-1         | *** Preparing to test memory region 4000202000 (18939904 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4001c58000 (12288 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4001de0000 (8192 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4002168000 (16384 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4002189000 (16384 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 400234a000 (40960 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4002600000 (2097152 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4002954000 (2097152 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4003000000 (4194304 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4003593000 (8388608 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4003d94000 (8388608 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4004595000 (8388608 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4004d96000 (8388608 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4005596000 (204472320 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4011896000 (1048576 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4011996000 (41943040 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4014196000 (4194304 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4014596000 (50331648 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4017800000 (2097152 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 4017b96000 (8388608 bytes)
compose-redis_roaring-1         | *** Preparing to test memory region 401c000000 (135168 bytes)
compose-redis_roaring-1         | .O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O
compose-redis_roaring-1         | Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.
compose-redis_roaring-1         |
compose-redis_roaring-1         | ------ DUMPING CODE AROUND EIP ------
compose-redis_roaring-1         | Symbol: dictAddRaw (base: 0x4000047ad0)
compose-redis_roaring-1         | Module: /usr/local/bin/redis-server *:6379 (base 0x4000000000)
compose-redis_roaring-1         | $ xxd -r -p /tmp/dump.hex /tmp/dump.bin
compose-redis_roaring-1         | $ objdump --adjust-vma=0x4000047ad0 -D -b binary -m i386:x86-64 /tmp/dump.bin
compose-redis_roaring-1         | ------
compose-redis_roaring-1         | 1:S 22 Apr 2022 09:37:01.949 # dump of function (hexdump of 491 bytes):
compose-redis_roaring-1         | 415741564989fe41554989d5415455534889f34883ec2848837f50ff740b66837f58000f846f010000498b064889dfff104889c54d85ed740849c74500000000004d8b4e504983f9ff0f85b10000004d8b4618498b7e284d85c00f84d00100004939f80f87970000008b05b1dd1a0085c0750e4889f831d249f7f04883f805767f498b06488d7701488b50304885d20f84d70100004885ff0f8812020000660fefc0f2480f2ac74d85c00f88e0010000660fefc9f2490f2ac848b8feffffffffffff7f31fff20f5ec14839c677224883fe040f860a020000bf04000000eb04904889c7488d043f4839f072f448c1e704ffd285c00f8566010000660f1f440000498d461048894424184989c1498d465048894424104d8b79104921ef4a8d04fd000000004889442408498b014e8b24f84d85e47536eb79660f1f840000000000498b06488b40184885c074154c890c24498b7e084889deffd04c8b0c2485c075134d8b6424104d85e47445498b14244839d375cc4d85ed74044d8965004531ed4883c4284c89e85b5d415c415d415e415fc3660f1f440000be01000000e80efaffffe982feffff660f1f84000000000049837e50ff0f84130100004983c1204c394c24100f854bffffff4983ffff74ad498d6e30bf18000000e872e700004c8b6424084c0365004989c549
compose-redis_roaring-1         | Function at 0x4000047680 is dictRehash
compose-redis_roaring-1         | Function at 0x4000056420 is zmalloc
compose-redis_roaring-1         |
compose-redis_roaring-1         | === REDIS BUG REPORT END. Make sure to include from START to END. ===
compose-redis_roaring-1         |
compose-redis_roaring-1         |        Please report the crash by opening an issue on github:
compose-redis_roaring-1         |
compose-redis_roaring-1         |            http://github.com/redis/redis/issues
compose-redis_roaring-1         |
compose-redis_roaring-1         |   Suspect RAM error? Use redis-server --test-memory to verify it.
compose-redis_roaring-1         |
compose-redis_roaring-1         | qemu: uncaught target signal 11 (Segmentation fault) - core dumped
aviggiano commented 2 years ago

Hi @BenjiLewis Thank you for opening this issue. I think it wouldn't be very hard to submit a fix for the docker image. Unfortunately lately I'm not having much time to update the project, so I'll take some time to take a look at this. If you want to submit a PR, I'd be happy to merge it to the codebase. Otherwise, I'll ping you when this issue is resolved in a while

BenjiLewis commented 2 years ago

Thanks for the quick response @aviggiano! I will take it from here and ping you when I have a PR.

BenjiLewis commented 2 years ago

I've built a docker image for cross platform builds https://hub.docker.com/r/benjilewis/redis-roaring

I would be happy to push to your account but decided to keep it isolated to avoid conflicts with architectures that I can't test on.