search

RedisLabsModules / redismod

A Docker image with select Redis Labs modules
BSD 3-Clause "New" or "Revised" License
127 stars 28 forks source link

ARM Support #49

Open mariusmuntean opened 1 year ago

mariusmuntean commented 1 year ago

Hi and thanks for this image! I found myself needing Redis+RediSearch+Redis JSON+Gears and this image was a godsent.

My only issue is that on my local machine, an M1 Macbook Pro, RedisMod crashes pretty reproducibly, whereas on an Intel Macbook Pro it runs as expected. I think it has to do with the fact that there's no native ARM build.

I'm running it like this

docker run \
  -p 6379:6379 \
  -v `pwd`:/data \
  -v `pwd`/docker-redis.conf:/usr/local/etc/redis/redis.conf \
  --platform linux/amd64 \
  redislabs/redismod \
  /usr/local/etc/redis/redis.conf

Notice the --platform linux/amd64 \ which I have to specify, otherwise I get

WARNING: The requested image's platform (linux/amd64) does not match the detected host platform (linux/arm64/v8) and no specific platform was requested
docker: Error response from daemon: platform linux/arm64/v8 not supported.

The docker-redis.conf looks like this

# Persistence defaults
save 60 5000

# Load necessary modules
loadmodule /usr/lib/redis/modules/rejson.so
loadmodule /usr/lib/redis/modules/redisearch.so
loadmodule /usr/lib/redis/modules/redisgraph.so
loadmodule /usr/lib/redis/modules/redisgears.so Plugin /var/opt/redislabs/modules/rg/plugin/gears_python.so

# Generated by CONFIG REWRITE
protected-mode no
user default on nopass ~* &* +@all

In my application I'm using Redis Gears to add a new property to all my Json documents in Redis (a few hundred documents) and then I'm updating the RedisSearch index to include that field.

It always fails with this error on my M1 Macbook Pro:

``` === REDIS BUG REPORT START: Cut & paste starting from here === 1:M 08 Jan 2023 15:48:00.577 # Redis 6.2.6 crashed by signal: 11, si_code: 1 1:M 08 Jan 2023 15:48:00.577 # Accessing address: 0x620008 1:M 08 Jan 2023 15:48:00.577 # Crashed running the instruction at: 0x4005eae9c4 ------ STACK TRACE ------ EIP: /usr/lib/redis/modules/redisgraph.so(DocTable_Free+0x34)[0x4005eae9c4] Backtrace: /lib/x86_64-linux-gnu/libpthread.so.0(+0x14140)[0x4002158140] /usr/lib/redis/modules/redisgraph.so(DocTable_Free+0x34)[0x4005eae9c4] /usr/lib/redis/modules/redisgraph.so(IndexSpec_FreeInternals+0x9b)[0x4005e52c8b] /usr/lib/redis/modules/redisgraph.so(RediSearch_DropIndex+0x13)[0x4005e4dc13] /usr/lib/redis/modules/redisgraph.so(Index_Construct+0x1f)[0x4004d5c8cf] /usr/lib/redis/modules/redisgraph.so(+0x1f507f)[0x4004d2407f] /usr/lib/redis/modules/redisgraph.so(+0x245da4)[0x4004d74da4] /lib/x86_64-linux-gnu/libpthread.so.0(+0x8ea7)[0x400214cea7] /lib/x86_64-linux-gnu/libc.so.6(clone+0x3f)[0x4002263def] ------ REGISTERS ------ 1:M 08 Jan 2023 15:48:00.581 # RAX:0000000000002780 RBX:0000000000620000 RCX:0000000000000001 RDX:0000000000000003 RDI:000000000061ffc8 RSI:0000004000000000 RBP:0000000000000278 RSP:000000400a4b5430 R8 :0000000000000128 R9 :0000004002600900 R10:000000400a4b5ec8 R11:0000000000000001 R12:000000403628b208 R13:000000405109c000 R14:000000400e1bcaa0 R15:0000004036281000 RIP:0000004005eae9c4 EFL:0000000000000206 CSGSFS:002b000000000033 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b543f) -> 0000000000000000 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b543e) -> 000000401a827310 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b543d) -> 0000004051092038 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b543c) -> 000000405109c000 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b543b) -> 0000004004d5c8cf 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b543a) -> 0000004051092038 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5439) -> 0000004005e4dc13 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5438) -> 000000400e1bcaa0 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5437) -> 000000405109c000 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5436) -> 000000403f295038 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5435) -> 000000401a827310 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5434) -> 000000403628b180 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5433) -> 0000004005e52c8b 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5432) -> 000000403f295038 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5431) -> 000000403628b180 1:M 08 Jan 2023 15:48:00.581 # (000000400a4b5430) -> 000000400646dbc0 ------ INFO OUTPUT ------ # Server redis_version:6.2.6 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:9c335ca9779faba5 redis_mode:standalone os:Linux 5.15.49-linuxkit x86_64 arch_bits:64 multiplexing_api:epoll atomicvar_api:atomic-builtin gcc_version:10.2.1 process_id:1 process_supervised:no run_id:44be8be77c77ec3c262115a20f0a977abade217c tcp_port:6379 server_time_usec:1673192880574820 uptime_in_seconds:16 uptime_in_days:0 hz:10 configured_hz:10 lru_clock:12248496 executable:/data/redis-server config_file:/usr/local/etc/redis/redis.conf io_threads_active:0 # Clients connected_clients:10 cluster_connections:0 maxclients:10000 client_recent_max_input_buffer:455 client_recent_max_output_buffer:0 blocked_clients:1 tracking_clients:0 clients_in_timeout_table:0 # Memory used_memory:16755232 used_memory_human:15.98M used_memory_rss:0 used_memory_rss_human:0B used_memory_peak:16831848 used_memory_peak_human:16.05M used_memory_peak_perc:99.54% used_memory_overhead:9480206 used_memory_startup:9243472 used_memory_dataset:7275026 used_memory_dataset_perc:96.85% allocator_allocated:17660248 allocator_active:19320832 allocator_resident:22544384 total_system_memory:4124385280 total_system_memory_human:3.84G used_memory_lua:37888 used_memory_lua_human:37.00K used_memory_scripts:0 used_memory_scripts_human:0B number_of_cached_scripts:0 maxmemory:0 maxmemory_human:0B maxmemory_policy:noeviction allocator_frag_ratio:1.09 allocator_frag_bytes:1660584 allocator_rss_ratio:1.17 allocator_rss_bytes:3223552 rss_overhead_ratio:0.00 rss_overhead_bytes:-22544384 mem_fragmentation_ratio:0.00 mem_fragmentation_bytes:-16619856 mem_not_counted_for_evict:0 mem_replication_backlog:0 mem_clients_slaves:0 mem_clients_normal:205110 mem_aof_buffer:0 mem_allocator:jemalloc-5.1.0 active_defrag_running:0 lazyfree_pending_objects:0 lazyfreed_objects:0 # Persistence loading:0 current_cow_size:0 current_cow_size_age:0 current_fork_perc:0.00 current_save_keys_processed:0 current_save_keys_total:0 rdb_changes_since_last_save:585 rdb_bgsave_in_progress:0 rdb_last_save_time:1673192864 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:-1 rdb_current_bgsave_time_sec:-1 rdb_last_cow_size:0 aof_enabled:0 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_last_write_status:ok aof_last_cow_size:0 module_fork_in_progress:0 module_fork_last_cow_size:0 # Stats total_connections_received:10 total_commands_processed:1235 instantaneous_ops_per_sec:1 total_net_input_bytes:3726 total_net_output_bytes:13122 instantaneous_input_kbps:0.11 instantaneous_output_kbps:0.07 rejected_connections:0 sync_full:0 sync_partial_ok:0 sync_partial_err:0 expired_keys:0 expired_stale_perc:0.00 expired_time_cap_reached_count:0 expire_cycle_cpu_milliseconds:1 evicted_keys:0 keyspace_hits:1761 keyspace_misses:5 pubsub_channels:1 pubsub_patterns:0 latest_fork_usec:0 total_forks:0 migrate_cached_sockets:0 slave_expires_tracked_keys:0 active_defrag_hits:0 active_defrag_misses:0 active_defrag_key_hits:0 active_defrag_key_misses:0 tracking_total_keys:0 tracking_total_items:0 tracking_total_prefixes:0 unexpected_error_replies:0 total_error_replies:10 dump_payload_sanitizations:0 total_reads_processed:22 total_writes_processed:25 io_threaded_reads_processed:0 io_threaded_writes_processed:0 # Replication role:master connected_slaves:0 master_failover_state:no-failover master_replid:6c1515e51f11e042170112e9ec841acaf04cae49 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0 # CPU used_cpu_sys:0.214587 used_cpu_user:1.225623 used_cpu_sys_children:0.000000 used_cpu_user_children:0.000000 used_cpu_sys_main_thread:0.008020 used_cpu_user_main_thread:0.178458 # Modules module:name=rg,ver=10204,api=1,filters=1,usedby=[rg],using=[rg],options=[] module:name=graph,ver=20815,api=1,filters=0,usedby=[],using=[ReJSON],options=[] module:name=search,ver=999999,api=1,filters=0,usedby=[],using=[ReJSON],options=[handle-io-errors] module:name=ReJSON,ver=20011,api=1,filters=0,usedby=[search|graph],using=[],options=[handle-io-errors] # Commandstats cmdstat_rg.pyexecute:calls=1,usec=256707,usec_per_call=256707.00,rejected_calls=0,failed_calls=0 cmdstat_graph.QUERY:calls=5,usec=206738,usec_per_call=41347.60,rejected_calls=0,failed_calls=0 cmdstat_select:calls=4,usec=222,usec_per_call=55.50,rejected_calls=0,failed_calls=0 cmdstat_info:calls=17,usec=3251,usec_per_call=191.24,rejected_calls=0,failed_calls=0 cmdstat_config:calls=11,usec=2438,usec_per_call=221.64,rejected_calls=0,failed_calls=0 cmdstat_cluster:calls=5,usec=180,usec_per_call=36.00,rejected_calls=0,failed_calls=5 cmdstat_echo:calls=10,usec=248,usec_per_call=24.80,rejected_calls=0,failed_calls=0 cmdstat_get:calls=5,usec=447,usec_per_call=89.40,rejected_calls=0,failed_calls=0 cmdstat_json.get:calls=581,usec=16688,usec_per_call=28.72,rejected_calls=0,failed_calls=0 cmdstat_json.set:calls=581,usec=31328,usec_per_call=53.92,rejected_calls=0,failed_calls=0 cmdstat_subscribe:calls=5,usec=574,usec_per_call=114.80,rejected_calls=0,failed_calls=0 cmdstat_client:calls=10,usec=434,usec_per_call=43.40,rejected_calls=0,failed_calls=0 # Errorstats errorstat_ERR:count=10 # Cluster cluster_enabled:0 # Keyspace db0:keys=585,expires=0,avg_ttl=0 ------ CLIENT LIST OUTPUT ------ id=15 addr=172.17.0.1:60934 laddr=172.17.0.2:6379 fd=16 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=7 idle=0 flags=b db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=40954 argv-mem=69 obl=0 oll=0 omem=0 tot-mem=61541 events=r cmd=graph.QUERY user=default redir=-1 id=16 addr=172.17.0.1:60950 laddr=172.17.0.2:6379 fd=17 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=7 idle=6 flags=P db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=subscribe user=default redir=-1 id=17 addr=172.17.0.1:60966 laddr=172.17.0.2:6379 fd=18 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=6 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=info user=default redir=-1 id=18 addr=172.17.0.1:60970 laddr=172.17.0.2:6379 fd=19 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=6 flags=P db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=subscribe user=default redir=-1 id=19 addr=172.17.0.1:60986 laddr=172.17.0.2:6379 fd=20 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=6 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=info user=default redir=-1 id=20 addr=172.17.0.1:61000 laddr=172.17.0.2:6379 fd=21 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=6 flags=P db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=subscribe user=default redir=-1 id=21 addr=172.17.0.1:61014 laddr=172.17.0.2:6379 fd=22 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=6 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=info user=default redir=-1 id=22 addr=172.17.0.1:61022 laddr=172.17.0.2:6379 fd=23 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=6 flags=P db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=subscribe user=default redir=-1 id=23 addr=172.17.0.1:61034 laddr=172.17.0.2:6379 fd=24 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=3 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=rg.pyexecute user=default redir=-1 id=24 addr=172.17.0.1:61036 laddr=172.17.0.2:6379 fd=25 name=Mariuss-M-Pro(SE.Redis-v2.6.48.48654) age=6 idle=6 flags=P db=0 sub=1 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=0 omem=0 tot-mem=20504 events=r cmd=subscribe user=default redir=-1 ------ MODULES INFO OUTPUT ------ # rg rg_nexecutions:0 rg_nregistrations:0 # rg_regisrations # rg_plugins rg_GearsPythonPlugin:version=10000 # rg_python_stats rg_TotalAllocated:31961672 rg_PeakAllocated:8576320 rg_CurrAllocated:8515464 # rg_python_requirements # rg_python_sessions # graph_executing commands graph_command:GRAPH.QUERY CREATE INDEX ON :mentioned(RelationshipType) # ReJSON_trace ReJSON_trace: 0: redis_module::base_info_func 1: rejson::__info_func 2: modulesCollectInfo at /usr/src/redis/src/module.c:7078:9 3: logModulesInfo at /usr/src/redis/src/debug.c:1598:22 4: printCrashReport at /usr/src/redis/src/debug.c:1849:5 sigsegvHandler at /usr/src/redis/src/debug.c:1831:5 5: 6: DocTable_Free 7: IndexSpec_FreeInternals 8: RediSearch_DropIndex 9: Index_Construct 10: _ExecuteQuery 11: thread_do 12: start_thread 13: clone ------ FAST MEMORY TEST ------ 1:M 08 Jan 2023 15:48:00.636 # main thread terminated 1:M 08 Jan 2023 15:48:00.637 # Bio thread for job type #0 terminated 1:M 08 Jan 2023 15:48:00.637 # Bio thread for job type #1 terminated 1:M 08 Jan 2023 15:48:00.637 # Bio thread for job type #2 terminated Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible. ------ DUMPING CODE AROUND EIP ------ Symbol: DocTable_Free (base: 0x4005eae990) Module: /usr/lib/redis/modules/redisgraph.so (base 0x4004b2f000) $ xxd -r -p /tmp/dump.hex /tmp/dump.bin $ objdump --adjust-vma=0x4005eae990 -D -b binary -m i386:x86-64 /tmp/dump.bin ------ 1:M 08 Jan 2023 15:48:00.639 # dump of function (hexdump of 180 bytes): 41544989fc5549837c24180053488b7f30743f31ed0f1f004889e848c1e004488b1c074885db741f0f1f840000000000488d7bc8488b5b08e8c3edffff4885db75ee498b7c24304883c50149396c241877c6488b0517d95600ff105b498d7c24385d415ce987ffffff0f1f8000000000488b3f488d0db6e9ffff0fb7d2e9fe21feff0f1f4000662e0f1f840000000000415641554989d641544c8d673855534889fd4989f54c89e7e833f5ffff4885c00f841a01 Function at 0x4005ead790 is DMD_Free Function at 0x4005eadf70 is DocIdMap_Get === REDIS BUG REPORT END. Make sure to include from START to END. === ```
epinzur commented 1 year ago

I'm seeing similar issues using RedisAI on apple silicon. It would be great to get ARM support for RedisAI.

DjellalAbdou commented 1 year ago

am having the same issue, is there a workaround for it until ARM support is added?