search

danitseitlin / redis-interval-sets

Redis module for creating interval sets, handling them and filtering out relevant sets by number in range.
BSD 3-Clause "New" or "Revised" License
20 stars 2 forks source link

Redis crashing using module #20

Closed stopforumspam closed 2 years ago

stopforumspam commented 3 years ago

`2781896:M 03 Jun 2021 02:04:06.846 # Redis 255.255.255 crashed by signal: 11, si_code: 1 2781896:M 03 Jun 2021 02:04:06.846 # Accessing address: (nil)

------ STACK TRACE ------

Backtrace: /lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0)[0x7fbdb76423c0]

------ REGISTERS ------ 2781896:M 03 Jun 2021 02:04:06.846 # RAX:0000000000000009 RBX:00007fbdb6e22080 RCX:0000000000000000 RDX:00007ffc6c639120 RDI:00007ffc6c638e10 RSI:00007fbdb6e4d0d0 RBP:00007ffc6c6391a0 RSP:00007ffc6c638df8 R8 :0000000000000008 R9 :00007ffc6c638d00 R10:0000000000000068 R11:00007fbdb7000900 R12:00007fbdb6ea5f10 R13:00007ffc6c639120 R14:0000000000000000 R15:0000000000000000 RIP:0000000000000000 EFL:0000000000010217 CSGSFS:002b000000000033 2781896:M 03 Jun 2021 02:04:06.846 # (00007ffc6c638e07) -> 0000000000000008 2781896:M 03 Jun 2021 02:04:06.846 # (00007ffc6c638e06) -> 0000007000000101 2781896:M 03 Jun 2021 02:04:06.846 # (00007ffc6c638e05) -> 0000000000000040 2781896:M 03 Jun 2021 02:04:06.847 # (00007ffc6c638e04) -> 0000000000001030 2781896:M 03 Jun 2021 02:04:06.847 # (00007ffc6c638e03) -> 0000564515ebb808 2781896:M 03 Jun 2021 02:04:06.847 # (00007ffc6c638e02) -> 00007ffc00000063 2781896:M 03 Jun 2021 02:04:06.847 # (00007ffc6c638e01) -> 0000000000001000 2781896:M 03 Jun 2021 02:04:06.847 # (00007ffc6c638e00) -> 00007ffc6c639120 2781896:M 03 Jun 2021 02:04:06.847 # (00007ffc6c638dff) -> 0000000000000000 2781896:M 03 Jun 2021 02:04:06.847 # (00007ffc6c638dfe) -> 0000000000000000 2781896:M 03 Jun 2021 02:04:06.847 # (00007ffc6c638dfd) -> 00007fbdb6e22080 2781896:M 03 Jun 2021 02:04:06.847 # (00007ffc6c638dfc) -> 00007ffc6c6391a0 2781896:M 03 Jun 2021 02:04:06.847 # (00007ffc6c638dfb) -> 0000000000000009 2781896:M 03 Jun 2021 02:04:06.847 # (00007ffc6c638dfa) -> 00007ffc6c639120 2781896:M 03 Jun 2021 02:04:06.847 # (00007ffc6c638df9) -> 00007ffc6c63913e 2781896:M 03 Jun 2021 02:04:06.847 # (00007ffc6c638df8) -> 0000564515853477

------ INFO OUTPUT ------

Server

redis_version:255.255.255 redis_git_sha1:a049f629 redis_git_dirty:0 redis_build_id:f789c5cf2d7c81a8 redis_mode:standalone os:Linux 5.4.0-70-generic x86_64 arch_bits:64 multiplexing_api:epoll atomicvar_api:c11-builtin gcc_version:9.3.0 process_id:2781896 process_supervised:no run_id:1b2fc434d8343e97f3887fc1655bd3b199c64462 tcp_port:6379 server_time_usec:1622685846846416 uptime_in_seconds:106 uptime_in_days:0 hz:10 configured_hz:10 lru_clock:12073110 executable:/usr/bin/redis-server config_file:/etc/redis/redis.conf io_threads_active:0

Clients

connected_clients:2 cluster_connections:0 maxclients:10000 client_recent_max_input_buffer:48 client_recent_max_output_buffer:2665520 blocked_clients:0 tracking_clients:0 clients_in_timeout_table:0

Memory

used_memory:15222824 used_memory_human:14.52M used_memory_rss:17379328 used_memory_rss_human:16.57M used_memory_peak:19737152 used_memory_peak_human:18.82M used_memory_peak_perc:77.13% used_memory_overhead:3353152 used_memory_startup:810568 used_memory_dataset:11869672 used_memory_dataset_perc:82.36% allocator_allocated:15557712 allocator_active:16375808 allocator_resident:19271680 total_system_memory:16790601728 total_system_memory_human:15.64G used_memory_lua:37888 used_memory_lua_human:37.00K used_memory_scripts:0 used_memory_scripts_human:0B number_of_cached_scripts:0 maxmemory:0 maxmemory_human:0B maxmemory_policy:noeviction allocator_frag_ratio:1.05 allocator_frag_bytes:818096 allocator_rss_ratio:1.18 allocator_rss_bytes:2895872 rss_overhead_ratio:0.90 rss_overhead_bytes:-1892352 mem_fragmentation_ratio:1.14 mem_fragmentation_bytes:2199320 mem_not_counted_for_evict:0 mem_replication_backlog:0 mem_clients_slaves:0 mem_clients_normal:2542512 mem_aof_buffer:0 mem_allocator:jemalloc-5.1.0 active_defrag_running:0 lazyfree_pending_objects:0 lazyfreed_objects:0

Persistence

loading:0 current_cow_size:0 current_cow_size_age:0 current_fork_perc:0.00 current_save_keys_processed:0 current_save_keys_total:0 rdb_changes_since_last_save:1 rdb_bgsave_in_progress:0 rdb_last_save_time:1622685787 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:-1 rdb_current_bgsave_time_sec:-1 rdb_last_cow_size:0 aof_enabled:0 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_last_write_status:ok aof_last_cow_size:0 module_fork_in_progress:0 module_fork_last_cow_size:0

Stats

total_connections_received:2 total_commands_processed:447419 instantaneous_ops_per_sec:0 total_net_input_bytes:36275763 total_net_output_bytes:9035623 instantaneous_input_kbps:0.01 instantaneous_output_kbps:12.21 rejected_connections:0 sync_full:0 sync_partial_ok:0 sync_partial_err:0 expired_keys:0 expired_stale_perc:0.00 expired_time_cap_reached_count:0 expire_cycle_cpu_milliseconds:1 evicted_keys:0 keyspace_hits:0 keyspace_misses:0 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:0 total_forks:0 migrate_cached_sockets:0 slave_expires_tracked_keys:0 active_defrag_hits:0 active_defrag_misses:0 active_defrag_key_hits:0 active_defrag_key_misses:0 tracking_total_keys:0 tracking_total_items:0 tracking_total_prefixes:0 unexpected_error_replies:0 total_error_replies:280664 dump_payload_sanitizations:0 total_reads_processed:2218 total_writes_processed:560 io_threaded_reads_processed:0 io_threaded_writes_processed:0

Replication

role:master connected_slaves:0 master_failover_state:no-failover master_replid:4c03be0c6868d319686a6f617e80115f2454c272 master_replid2:0000000000000000000000000000000000000000 master_repl_offset:0 second_repl_offset:-1 repl_backlog_active:0 repl_backlog_size:0 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0

CPU

used_cpu_sys:0.557793 used_cpu_user:1.810808 used_cpu_sys_children:0.000000 used_cpu_user_children:0.000000 used_cpu_sys_main_thread:0.552498 used_cpu_user_main_thread:1.806709

Modules

module:name=interval_set,ver=1,api=1,filters=0,usedby=[],using=[],options=[]

Commandstats

cmdstat_iset.del:calls=1,usec=22,usec_per_call=22.00,rejected_calls=0,failed_calls=0 cmdstat_command:calls=1,usec=1485,usec_per_call=1485.00,rejected_calls=0,failed_calls=0 cmdstat_iset.add:calls=447416,usec=1623358,usec_per_call=3.63,rejected_calls=0,failed_calls=280664 cmdstat_flushall:calls=1,usec=2009,usec_per_call=2009.00,rejected_calls=0,failed_calls=0

Errorstats

errorstat_UTF8:count=280664

Cluster

cluster_enabled:0

Keyspace

db0:keys=1,expires=0,avg_ttl=0

------ CLIENT LIST OUTPUT ------ id=3 addr=127.0.0.1:49856 laddr=127.0.0.1:6379 fd=7 name= age=59 idle=4 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 argv-mem=0 obl=0 oll=122 omem=2501488 tot-mem=2522016 events=rw cmd=iset.add user=default redir=-1 id=4 addr=127.0.0.1:49858 laddr=127.0.0.1:6379 fd=8 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=14 qbuf-free=40940 argv-mem=4 obl=0 oll=0 omem=0 tot-mem=61452 events=r cmd=save user=default redir=-1

------ CURRENT CLIENT INFO ------ id=4 addr=127.0.0.1:49858 laddr=127.0.0.1:6379 fd=8 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=14 qbuf-free=40940 argv-mem=4 obl=0 oll=0 omem=0 tot-mem=61452 events=r cmd=save user=default redir=-1 argv[0]: 'save'

------ MODULES INFO OUTPUT ------

------ FAST MEMORY TEST ------ 2781896:M 03 Jun 2021 02:04:06.848 # Bio thread for job type #0 terminated 2781896:M 03 Jun 2021 02:04:06.848 # Bio thread for job type #1 terminated 2781896:M 03 Jun 2021 02:04:06.848 # Bio thread for job type #2 terminated 2781896:M 03 Jun 2021 02:04:06.848 # IO thread(tid:140452758603520) terminated 2781896:M 03 Jun 2021 02:04:06.849 # IO thread(tid:140452750210816) terminated 2781896:M 03 Jun 2021 02:04:06.849 # IO thread(tid:140452741818112) terminated Preparing to test memory region 5645159e8000 (2277376 bytes) Preparing to test memory region 564515ebb000 (135168 bytes) Preparing to test memory region 7fbdb0a89000 (35651584 bytes) Preparing to test memory region 7fbdb2c8a000 (8388608 bytes) Preparing to test memory region 7fbdb348b000 (8388608 bytes) Preparing to test memory region 7fbdb3c8c000 (8388608 bytes) Preparing to test memory region 7fbdb448d000 (8388608 bytes) Preparing to test memory region 7fbdb4c8e000 (8388608 bytes) Preparing to test memory region 7fbdb6a00000 (8388608 bytes) Preparing to test memory region 7fbdb729d000 (24576 bytes) Preparing to test memory region 7fbdb742e000 (8192 bytes) Preparing to test memory region 7fbdb7629000 (16384 bytes) Preparing to test memory region 7fbdb764c000 (16384 bytes) Preparing to test memory region 7fbdb76fe000 (4096 bytes) Preparing to test memory region 7fbdb7854000 (8192 bytes) Preparing to test memory region 7fbdb7899000 (4096 bytes) .O.O.O.O.O.O.O.O.O.O.O.O.O.O.O.O Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.

=== REDIS BUG REPORT END. Make sure to include from START to END. ===

   Please report the crash by opening an issue on github:

       http://github.com/redis/redis/issues

Suspect RAM error? Use redis-server --test-memory to verify it.

2782056:C 03 Jun 2021 02:04:07.906 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 2782056:C 03 Jun 2021 02:04:07.906 # Redis version=255.255.255, bits=64, commit=a049f629, modified=0, pid=2782056, just started 2782056:C 03 Jun 2021 02:04:07.906 # Configuration loaded 2782056:M 03 Jun 2021 02:04:07.907 * monotonic clock: POSIX clockgettime .
.-__ ''-._ _.- .. ''-. Redis 255.255.255 (a049f629/0) 64 bit .-.-```. ```\/ _.,_ ''-._ ( ' , .-` | `, ) Running in standalone mode |`-._`-...-` __...-.-.|'` .-'| Port: 6379 | -._. / .-' | PID: 2782056 -._-. `-./ .-' .-'
|`-.-._-..-' .-'.-'|
| -._-. .-'.-' | https://redis.io
`-. -._-..-'.-' .-'
|-._-._ -.__.-' _.-'_.-'| |-.`-. .-'.-' |
-._-._-.__.-'_.-' _.-' -._ -.__.-' _.-' -. .-'
`-.__.-'

2782056:M 03 Jun 2021 02:04:07.908 # Server initialized 2782056:M 03 Jun 2021 02:04:07.909 Created new data type 'IntervlSt' 2782056:M 03 Jun 2021 02:04:07.909 Module 'interval_set' loaded from /usr/local/bin/libintervalsets.so 2782056:M 03 Jun 2021 02:04:07.910 Loading RDB produced by version 255.255.255 2782056:M 03 Jun 2021 02:04:07.910 RDB age 60 seconds 2782056:M 03 Jun 2021 02:04:07.910 RDB memory usage when created 0.83 Mb 2782056:M 03 Jun 2021 02:04:07.910 DB loaded from disk: 0.000 seconds 2782056:M 03 Jun 2021 02:04:07.910 * Ready to accept connections`

This only happens with the interval set usage. The server itself is a E5 xeon with ECC memory which tests out fine. The redis binary was compiled on Ubuntu 20.04 x64 from the 6.23 release and the git pull from Jun 2nd 2021

I've replicated it with this small PHP script that pulls in ASN, country and IP data. Most of the time the script fails to execute. running up the cli and issuing a SAVE will generate the dump

`<?php

$redis = new Redis();

$redis->connect("127.0.0.1");

function ip2long32($ip) {

return ( ip2long($ip) & 0xFFFFFFFF );

}

$handle = fopen("ip2asn-combined.tsv", "r");

if (!$handle) {

die("error reading ip2asn-combined.tsv - this can be downloaded from https://iptoasn.com/data/ip2asn-combined.tsv.gz

\n");

}

$redis->flushall();

$m=$redis->pipeline();

echo "Reading ip data...\n";

while (($line = fgetcsv($handle, 0 , "\t")) !== false) {

// don't process ASN 0 / no routed addresses

if ($line[2] != "0") {

    $iptype = (strpos($line[0], '.') !== false ) ? 4 : 6;

    if ($iptype == 4) {

        $s = ip2long32($line[0]);

        $e = ip2long32($line[1]);

    }  else {

        $start = inet_pton($line[0]);

        $end = inet_pton($line[1]);

        // inet_pton returns a packed binary, so unpack, which unpacks into an array

        $s = unpack("H*", $start);

        $e = unpack("H*", $end);

        // top 48 bits = 6 bytes

        $s = hexdec(substr(array_pop($s), 0, 12));

        $e = hexdec(substr(array_pop($e), 0, 12));

    }

    $a = intval($line[2]);

    $a = ($a < 65536) ? pack('S', $a) :  pack('L', $a); //pack('Q', $a);

    $c = strtolower($line[3]);      

    $m->rawcommand("iset.add", "iset_geo:ip", $c . $a, $s, $e);

}

}

echo "Executing...";

$m->exec();

echo "Done...\n";

fclose($handle);`

Rerunning the script will randomly throw this exception as well

PHP Fatal error: Uncaught RedisException: MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error. in iset.php:24

stopforumspam commented 3 years ago

Most of my testing will never see "done" reported as redis crashes out

ps i love the module. ive been using hard coded sourcetree change version of this for several years

danitseitlin commented 3 years ago

@stopforumspam Hi, This log is quite big. Mind pointing out the exact command that is causing issues?

stopforumspam commented 3 years ago

It's really only one command, iset.add run 450,000 times in a large multi pipeline. The php script in the ticket that I used to generate the report. It will crash redis most of the time it runs, in libpthread it seems

From: Dani Tseitlin @.> Sent: Friday, June 4, 2021 6:16:57 PM To: danitseitlin/redis-interval-sets @.> Cc: stopforumspam @.>; Mention @.> Subject: Re: [danitseitlin/redis-interval-sets] Redis crashing using module (#20)

@stopforumspamhttps://github.com/stopforumspam Hi, This log is quite big. Mind pointing out the exact command that is causing issues?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/danitseitlin/redis-interval-sets/issues/20#issuecomment-854392584, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKT2M3OQWKUI4IJ2ZL6R4A3TRBVVTANCNFSM4572BRUA.

stopforumspam commented 3 years ago

I should add that the same code using zsets instead of isets will run continuously without error

From: Paul Lush @.> Sent: Friday, June 4, 2021 7:21:01 PM To: danitseitlin/redis-interval-sets @.>; danitseitlin/redis-interval-sets @.> Cc: Mention @.> Subject: Re: [danitseitlin/redis-interval-sets] Redis crashing using module (#20)

It's really only one command, iset.add run 450,000 times in a large multi pipeline. The php script in the ticket that I used to generate the report. It will crash redis most of the time it runs, in libpthread it seems

From: Dani Tseitlin @.> Sent: Friday, June 4, 2021 6:16:57 PM To: danitseitlin/redis-interval-sets @.> Cc: stopforumspam @.>; Mention @.> Subject: Re: [danitseitlin/redis-interval-sets] Redis crashing using module (#20)

@stopforumspamhttps://github.com/stopforumspam Hi, This log is quite big. Mind pointing out the exact command that is causing issues?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/danitseitlin/redis-interval-sets/issues/20#issuecomment-854392584, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AKT2M3OQWKUI4IJ2ZL6R4A3TRBVVTANCNFSM4572BRUA.

danitseitlin commented 3 years ago

@gavrie Any chance you have any insights of how to debug this issue?

gavrie commented 3 years ago

@danitseitlin the first step would be to create the simplest possible setup that reproduces the bug reliably. Have you succeeded in reproducing it? After that, it can be run against a debug build of the module and/or by running Redis under a debugger, and inspecting the stack trace for details. That usually points out the problem. Let me know if you'd like to discuss the details.

stopforumspam commented 3 years ago

I compiled a debug build and ran it in gdb, and guess what... no crash.

I'll try to replicate a crash in a debugger

stopforumspam commented 3 years ago

OK, a crash

`Count of items: 169386 Count of items before new item: 169386 Count of items: 169387 Count of items before new item: 169387 Count of items: 169388 Count of items before new item: 169388 Count of items: 169389 Count of items before new item: 169389 Count of items: 169390 Count of items before new item: 169390 Count of items: 169391 Count of items before new item: 169391 Count of items: 169392 Count of items before new item: 169392 Count of items: 169393 Count of items before new item: 169393 Count of items: 169394 Count of items before new item: 169394 Count of items: 169395 Count of items before new item: 169395 Count of items: 169396 Count of items before new item: 169396 Count of items: 169397

`

stopforumspam commented 3 years ago

it hangs there. if you run up another terminal and issue a save then it segfaults

stopforumspam commented 3 years ago

the lack of crashing was because i forgot to run gdb with the --args so it was ignoring redis.conf and therefore not loading the module. Sorry

stopforumspam commented 3 years ago

`... Count of items: 169392

Count of items before new item: 169392

Count of items: 169393

Count of items before new item: 169393

Count of items: 169394

Count of items before new item: 169394

Count of items: 169395

Count of items before new item: 169395

Count of items: 169396

Count of items before new item: 169396

Count of items: 169397`

the insert process hangs here, during the ->exec() , so I issue a save into another window and then get this

`Thread 1 "redis-server" received signal SIGSEGV, Segmentation fault.

0x0000000000000000 in ?? ()

(gdb) backtrace

0 0x0000000000000000 in ?? ()

1 0x00005555555d04d7 in rdbSaveObject (rdb=0x7fffffffd080, o=, key=0x7fffffffd000) at rdb.c:1046

2 0x00005555555d0d39 in rdbSaveKeyValuePair (rdb=rdb@entry=0x7fffffffd080, key=key@entry=0x7fffffffd000, val=val@entry=0x7ffff74a5d90, expiretime=) at rdb.c:1110

3 0x00005555555d14b8 in rdbSaveRio (rdb=0x7fffffffd080, error=0x7fffffffd07c, rdbflags=, rsi=0x0) at rdb.c:1261

4 0x00005555555d3cf3 in rdbSave (filename=0x7ffff744a530 "dump.rdb", rsi=0x0) at rdb.c:1377

5 0x00005555555d56fc in saveCommand (c=0x7ffff2f1d6c0) at rdb.c:2863

6 0x00005555555a2126 in call (c=0x7ffff2f1d6c0, flags=15) at server.c:3713

7 0x00005555555a2fc3 in processCommand (c=c@entry=0x7ffff2f1d6c0) at server.c:4238

8 0x00005555555b71d0 in processCommandAndResetClient (c=c@entry=0x7ffff2f1d6c0) at networking.c:2010

9 0x00005555555b9a4a in processInputBuffer (c=0x7ffff2f1d6c0) at networking.c:2111

10 0x000055555565109c in callHandler (handler=, conn=0x7ffff7415200) at connhelpers.h:79

11 connSocketEventHandler (el=, fd=, clientData=0x7ffff7415200, mask=) at connection.c:295

12 0x000055555559a80a in aeProcessEvents (eventLoop=eventLoop@entry=0x7ffff74230a0, flags=flags@entry=27) at ae.c:427

13 0x000055555559aa9d in aeMain (eventLoop=0x7ffff74230a0) at ae.c:487

14 0x0000555555596ed7 in main (argc=, argv=) at server.c:6392`

rdb.c:1046

`/ Then write the module-specific representation + EOF marker. /

mt->rdb_save(&io,mv->value);`

sp it looks like two issues are happening, the interval set insert is hanging and then redis is crashing while the insert is hanging when it goes to save a db

stopforumspam commented 3 years ago

I added more debug logging and is_add is completing every operation, returning REDIS_OK on every insert

I'm now thinking that this is a bug Redis that your code has happened upon

I removed the process from using a multi insert to just iterating over the CSV and everything completes processing without a crash.

What are your thoughts?

stopforumspam commented 3 years ago

further testing with the iteration vs multi still gets segfaults in save, first with this when issuing a flush

PHP Fatal error: Uncaught RedisException: MISCONF Redis is configured to save RDB snapshots, but it is currently not able to persist on disk. Commands that may modify the data set are disabled, because this instance is configured to report errors during writes if RDB snapshotting fails (stop-writes-on-bgsave-error option). Please check the Redis logs for details about the RDB error. when issuing a flushall()

and then the segfault when issuing a save after that, with the same backtrace

gavrie commented 3 years ago

Thanks @stopforumspam!

@danitseitlin I suggest that you try to replicate the crash and then we can take it from there. The fact that it works fine with zset leads me to think that the problem is in the module, not in Redis. My first guess would be that it's due to some resource constraint, so it may be worthwhile to use a memory profiler and see if there's a leak somewhere.

danitseitlin commented 3 years ago

@gavrie Thanks, i will contact you further for more help on the subject. @stopforumspam Thank you for the issue and all of the details.

stopforumspam commented 3 years ago

if you disable save then it doesnt seem to crash, and the script will complete however if you then issue a save command, redis will segfault

danitseitlin commented 2 years ago

@stopforumspam Hi, is this the error:

------ CURRENT CLIENT INFO ------
id=5 addr=127.0.0.1:49683 fd=8 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=14 qbuf-free=32754 obl=0 oll=0 omem=0 events=r cmd=save user=default
argv[0]: 'SAVE'
Segmentation fault (core dumped)

?

stopforumspam commented 2 years ago

So I've replicated it easily.

127.0.0.1:6379> flushdb
OK
127.0.0.1:6379> save
OK
127.0.0.1:6379> iset.add test ip 1 2
OK
127.0.0.1:6379> save
Could not connect to Redis at 127.0.0.1:6379: Connection refused
(0.57s)
not connected>

And the redis dump

$ ./redis-server --loadmodule ../redis-interval-sets/target/debug/libintervalsets.so
1122834:C 01 Jan 2022 06:46:46.050 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1122834:C 01 Jan 2022 06:46:46.050 # Redis version=255.255.255, bits=64, commit=39ced1af, modified=0, pid=1122834, just started
1122834:C 01 Jan 2022 06:46:46.050 # Configuration loaded
1122834:M 01 Jan 2022 06:46:46.051 * monotonic clock: POSIX clock_gettime
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 255.255.255 (39ced1af/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                  
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 1122834
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           https://redis.io       
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

1122834:M 01 Jan 2022 06:46:46.052 # Server initialized
1122834:M 01 Jan 2022 06:46:46.052 * <interval_set> Created new data type 'IntervlSt'
1122834:M 01 Jan 2022 06:46:46.052 * Module 'interval_set' loaded from ../redis-interval-sets/target/debug/libintervalsets.so
1122834:M 01 Jan 2022 06:46:46.053 * Loading RDB produced by version 255.255.255
1122834:M 01 Jan 2022 06:46:46.053 * RDB age 11 seconds
1122834:M 01 Jan 2022 06:46:46.053 * RDB memory usage when created 0.85 Mb
1122834:M 01 Jan 2022 06:46:46.053 * Done loading RDB, keys loaded: 0, keys expired: 0.
1122834:M 01 Jan 2022 06:46:46.053 * DB loaded from disk: 0.000 seconds
1122834:M 01 Jan 2022 06:46:46.053 * Ready to accept connections
1122834:M 01 Jan 2022 06:46:56.650 * DB saved on disk
[iset.add] Adding a new key 'test'

=== REDIS BUG REPORT START: Cut & paste starting from here ===
1122834:M 01 Jan 2022 06:47:02.447 # Redis 255.255.255 crashed by signal: 11, si_code: 1
1122834:M 01 Jan 2022 06:47:02.447 # Accessing address: (nil)

------ STACK TRACE ------

Backtrace:
/lib/x86_64-linux-gnu/libpthread.so.0(+0x153c0)[0x7f0d377e73c0]

------ REGISTERS ------
1122834:M 01 Jan 2022 06:47:02.447 # 
RAX:0000000000000009 RBX:00007f0d37030140
RCX:0000000000000000 RDX:00007ffc268c8110
RDI:00007ffc268c7df0 RSI:00007f0d3704fc90
RBP:00007ffc268c8190 RSP:00007ffc268c7dd8
R8 :0000000000000008 R9 :00007ffc268c7d00
R10:00000000000000b8 R11:00007f0d37002800
R12:00007f0d37058520 R13:00007ffc268c8110
R14:0000000000000000 R15:0000000000000000
RIP:0000000000000000 EFL:0000000000010217
CSGSFS:002b000000000033
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7de7) -> 000000630000006f
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7de6) -> 000000770000007c
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7de5) -> 0000000000000000
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7de4) -> 0000000000000001
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7de3) -> 0000000000000008
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7de2) -> 0000007000000101
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7de1) -> 0000000000000000
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7de0) -> 00007ffc268c8110
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7ddf) -> 0000000000000000
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7dde) -> 0000000000000000
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7ddd) -> 00007f0d37030140
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7ddc) -> 00007ffc268c8190
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7ddb) -> 0000000000000009
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7dda) -> 0000000000000000
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7dd9) -> 00007ffc268c8110
1122834:M 01 Jan 2022 06:47:02.447 # (00007ffc268c7dd8) -> 0000562a04ca6bfb

------ INFO OUTPUT ------
# Server
redis_version:255.255.255
redis_git_sha1:39ced1af
redis_git_dirty:0
redis_build_id:6a9cee280d44073d
redis_mode:standalone
os:Linux 5.4.0-90-generic x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:9.3.0
process_id:1122834
process_supervised:no
run_id:62c225cf52b4dd7f702f6939864f1e0ef6b205da
tcp_port:6379
server_time_usec:1641019622447456
uptime_in_seconds:16
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:13629670
executable:/home/coder/code/redis/src/./redis-server
config_file:
io_threads_active:0

# Clients
connected_clients:1
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:20480
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0

# Memory
used_memory:893400
used_memory_human:872.46K
used_memory_rss:7729152
used_memory_rss_human:7.37M
used_memory_peak:893400
used_memory_peak_human:872.46K
used_memory_peak_perc:100.19%
used_memory_overhead:890016
used_memory_startup:848816
used_memory_dataset:3384
used_memory_dataset_perc:7.59%
allocator_allocated:1052592
allocator_active:1261568
allocator_resident:4673536
total_system_memory:16790593536
total_system_memory_human:15.64G
used_memory_lua:37888
used_memory_vm_eval:37888
used_memory_lua_human:37.00K
used_memory_scripts_eval:0
number_of_cached_scripts:0
number_of_functions:0
used_memory_vm_functions:35840
used_memory_vm_total:73728
used_memory_vm_total_human:72.00K
used_memory_functions:168
used_memory_scripts:168
used_memory_scripts_human:168B
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.20
allocator_frag_bytes:208976
allocator_rss_ratio:3.70
allocator_rss_bytes:3411968
rss_overhead_ratio:1.65
rss_overhead_bytes:3055616
mem_fragmentation_ratio:8.67
mem_fragmentation_bytes:6837624
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_total_replication_buffers:0
mem_clients_slaves:0
mem_clients_normal:40960
mem_cluster_links:0
mem_aof_buffer:0
mem_allocator:jemalloc-5.2.1
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0

# Persistence
loading:0
async_loading:0
current_cow_peak:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1641019616
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
rdb_last_load_keys_expired:0
rdb_last_load_keys_loaded:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

# Stats
total_connections_received:1
total_commands_processed:3
instantaneous_ops_per_sec:0
total_net_input_bytes:110
total_net_output_bytes:73
instantaneous_input_kbps:0.03
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
evicted_clients:0
total_eviction_exceeded_time:0
current_eviction_exceeded_time:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
total_active_defrag_time:0
current_active_defrag_time:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:1
dump_payload_sanitizations:0
total_reads_processed:5
total_writes_processed:4
io_threaded_reads_processed:0
io_threaded_writes_processed:0

# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:b806ad2f4d782f3627f817a03ff6dce00dacabf4
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

# CPU
used_cpu_sys:0.015166
used_cpu_user:0.024094
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
used_cpu_sys_main_thread:0.014174
used_cpu_user_main_thread:0.024804

# Modules
module:name=interval_set,ver=1,api=1,filters=0,usedby=[],using=[],options=[]

# Commandstats
cmdstat_save:calls=1,usec=2544,usec_per_call=2544.00,rejected_calls=0,failed_calls=0
cmdstat_flushdb:calls=1,usec=17,usec_per_call=17.00,rejected_calls=0,failed_calls=0
cmdstat_iset.add:calls=1,usec=113,usec_per_call=113.00,rejected_calls=0,failed_calls=0

# Errorstats
errorstat_ERR:count=1

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=1,expires=0,avg_ttl=0

------ CLIENT LIST OUTPUT ------
id=4 addr=127.0.0.1:38414 laddr=127.0.0.1:6379 fd=8 name= age=10 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=14 qbuf-free=20460 argv-mem=4 multi-mem=0 obl=0 oll=0 omem=0 tot-mem=40972 events=r cmd=save user=default redir=-1 resp=2

------ CURRENT CLIENT INFO ------
id=4 addr=127.0.0.1:38414 laddr=127.0.0.1:6379 fd=8 name= age=10 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=14 qbuf-free=20460 argv-mem=4 multi-mem=0 obl=0 oll=0 omem=0 tot-mem=40972 events=r cmd=save user=default redir=-1 resp=2
argv[0]: 'save'

------ MODULES INFO OUTPUT ------

------ CONFIG DEBUG OUTPUT ------
io-threads-do-reads no
lazyfree-lazy-eviction no
lazyfree-lazy-expire no
lazyfree-lazy-server-del no
lazyfree-lazy-user-del no
lazyfree-lazy-user-flush no
repl-diskless-sync no
replica-read-only yes
activedefrag no
repl-diskless-load disabled
sanitize-dump-payload no
io-threads 1
list-compress-depth 0
proto-max-bulk-len 512mb
client-query-buffer-limit 1gb

------ FAST MEMORY TEST ------
1122834:M 01 Jan 2022 06:47:02.448 # Bio thread for job type #0 terminated
1122834:M 01 Jan 2022 06:47:02.449 # Bio thread for job type #1 terminated
1122834:M 01 Jan 2022 06:47:02.449 # Bio thread for job type #2 terminated
*** Preparing to test memory region 562a04e9e000 (2297856 bytes)
*** Preparing to test memory region 562a06fe5000 (135168 bytes)
*** Preparing to test memory region 7f0d3460f000 (8388608 bytes)
*** Preparing to test memory region 7f0d34e10000 (8388608 bytes)
*** Preparing to test memory region 7f0d35611000 (8388608 bytes)
*** Preparing to test memory region 7f0d35e12000 (8388608 bytes)
*** Preparing to test memory region 7f0d36c00000 (8388608 bytes)
*** Preparing to test memory region 7f0d37442000 (24576 bytes)
*** Preparing to test memory region 7f0d375d3000 (8192 bytes)
*** Preparing to test memory region 7f0d377ce000 (16384 bytes)
*** Preparing to test memory region 7f0d377f1000 (16384 bytes)
*** Preparing to test memory region 7f0d378a3000 (4096 bytes)
*** Preparing to test memory region 7f0d379f9000 (8192 bytes)
*** Preparing to test memory region 7f0d37a46000 (4096 bytes)
.O.O.O.O.O.O.O.O.O.O.O.O.O.O
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.

=== REDIS BUG REPORT END. Make sure to include from START to END. ===

       Please report the crash by opening an issue on github:

           http://github.com/redis/redis/issues

  Suspect RAM error? Use redis-server --test-memory to verify it.

Segmentation fault (core dumped)```

Redis as of 1/1/2022

```commit 39ced1afa1ce90ac74116c156703beb66f0e211f (HEAD -> unstable, origin/unstable, origin/HEAD)
Author: Madelyn Olson <34459052+madolson@users.noreply.github.com>
Date:   Fri Dec 31 09:32:33 2021 -0800```

On Ubuntu 20.04 

```$ gcc -v
Using built-in specs.
COLLECT_GCC=gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/9/lto-wrapper
OFFLOAD_TARGET_NAMES=nvptx-none:hsa
OFFLOAD_TARGET_DEFAULT=1
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 9.3.0-17ubuntu1~20.04' --with-bugurl=file:///usr/share/doc/gcc-9/README.Bugs --enable-languages=c,ada,c++,go,brig,d,fortran,objc,obj-c++,gm2 --prefix=/usr --with-gcc-major-version-only --program-suffix=-9 --program-prefix=x86_64-linux-gnu- --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --libdir=/usr/lib --enable-nls --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --with-default-libstdcxx-abi=new --enable-gnu-unique-object --disable-vtable-verify --enable-plugin --enable-default-pie --with-system-zlib --with-target-system-zlib=auto --enable-objc-gc=auto --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --enable-multilib --with-tune=generic --enable-offload-targets=nvptx-none=/build/gcc-9-HskZEa/gcc-9-9.3.0/debian/tmp-nvptx/usr,hsa --without-cuda-driver --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 9.3.0 (Ubuntu 9.3.0-17ubuntu1~20.04)

and rust

$ rustc -V
rustc 1.53.0
stopforumspam commented 2 years ago

and from docker on ubuntu 20.04 running the same flush/iadd/save

$ docker run -p 6379:6379 --name ris danitseitlin/redis-interval-sets:latest
Unable to find image 'danitseitlin/redis-interval-sets:latest' locally
latest: Pulling from danitseitlin/redis-interval-sets
45b42c59be33: Pull complete 
5ce2e937bf62: Pull complete 
2a031498ff58: Pull complete 
2f3d47096658: Pull complete 
04f5cb8ac4c0: Pull complete 
9ed141398658: Pull complete 
54003d1f9d53: Pull complete 
cda04fd2d3a9: Pull complete 
Digest: sha256:2cf455e57dab75061b03ae0468ebe073069e8567c6dd5853f8dbc5ad0dfaf238
Status: Downloaded newer image for danitseitlin/redis-interval-sets:latest
1:C 01 Jan 2022 07:02:07.971 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 01 Jan 2022 07:02:07.971 # Redis version=6.2.1, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 01 Jan 2022 07:02:07.971 # Configuration loaded
1:M 01 Jan 2022 07:02:07.973 * monotonic clock: POSIX clock_gettime
1:M 01 Jan 2022 07:02:07.973 * Running mode=standalone, port=6379.
1:M 01 Jan 2022 07:02:07.974 # Server initialized
1:M 01 Jan 2022 07:02:07.974 * <interval_set> Created new data type 'IntervlSt'
1:M 01 Jan 2022 07:02:07.974 * Module 'interval_set' loaded from /usr/lib/redis/modules/ris.so
1:M 01 Jan 2022 07:02:07.975 * Ready to accept connections
Creating a new key
Count of items: 1

=== REDIS BUG REPORT START: Cut & paste starting from here ===
1:M 01 Jan 2022 07:02:35.472 # Redis 6.2.1 crashed by signal: 11, si_code: 1
1:M 01 Jan 2022 07:02:35.472 # Accessing address: (nil)
1:M 01 Jan 2022 07:02:35.472 # Killed by PID: 0, UID: 0

------ STACK TRACE ------

Backtrace:
/lib/x86_64-linux-gnu/libpthread.so.0(+0x12730)[0x7fbb7793e730]

------ REGISTERS ------
1:M 01 Jan 2022 07:02:35.472 # 
RAX:0000000000000009 RBX:00007fbb77222080
RCX:00b452f9ae5e7b22 RDX:00007ffd7fd53ed0
RDI:00007ffd7fd53c00 RSI:00007fbb7733fb48
RBP:00007fbb7727be20 RSP:00007ffd7fd53be8
R8 :000055ea986a6800 R9 :00007ffd7fd53bd0
R10:00000000000000e6 R11:00007fbb778d53e0
R12:00007ffd7fd53f30 R13:00007fbb7727bf40
R14:0000000000000000 R15:00007ffd7fd53ed0
RIP:0000000000000000 EFL:0000000000010217
CSGSFS:002b000000000033
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53bf7) -> ffffffffffffe760
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53bf6) -> 00000000000000ff
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53bf5) -> 0000000000001000
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53bf4) -> 00007fbb77924c40
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53bf3) -> 00007fbb77765730
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53bf2) -> 0000000000000000
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53bf1) -> 000000770000007c
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53bf0) -> 00007ffd7fd53ed0
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53bef) -> 0000000000000000
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53bee) -> 0000000000000000
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53bed) -> 00007fbb77222080
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53bec) -> 00007ffd7fd53f30
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53beb) -> 0000000000000009
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53bea) -> 0000000000001030
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53be9) -> 00007ffd7fd53ed0
1:M 01 Jan 2022 07:02:35.472 # (00007ffd7fd53be8) -> 000055ea97df19ed

------ INFO OUTPUT ------
# Server
redis_version:6.2.1
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:a26c8b6d79010f4f
redis_mode:standalone
os:Linux 5.4.0-90-generic x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:c11-builtin
gcc_version:8.3.0
process_id:1
process_supervised:no
run_id:063da46eb035a63ec2ba60b1524e509bdd36a0e2
tcp_port:6379
server_time_usec:1641020555471731
uptime_in_seconds:28
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:13630603
executable:/data/redis-server
config_file:
io_threads_active:0

# Clients
connected_clients:1
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:48
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0

# Memory
used_memory:875024
used_memory_human:854.52K
used_memory_rss:8589312
used_memory_rss_human:8.19M
used_memory_peak:934360
used_memory_peak_human:912.46K
used_memory_peak_perc:93.65%
used_memory_overhead:830448
used_memory_startup:809848
used_memory_dataset:44576
used_memory_dataset_perc:68.39%
allocator_allocated:1213848
allocator_active:1445888
allocator_resident:4304896
total_system_memory:16790593536
total_system_memory_human:15.64G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.19
allocator_frag_bytes:232040
allocator_rss_ratio:2.98
allocator_rss_bytes:2859008
rss_overhead_ratio:2.00
rss_overhead_bytes:4284416
mem_fragmentation_ratio:10.32
mem_fragmentation_bytes:7757072
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_clients_slaves:0
mem_clients_normal:20528
mem_aof_buffer:0
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0

# Persistence
loading:0
current_cow_size:0
current_fork_perc:0.00%
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1641020527
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

# Stats
total_connections_received:1
total_commands_processed:4
instantaneous_ops_per_sec:0
total_net_input_bytes:112
total_net_output_bytes:24780
instantaneous_input_kbps:0.03
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:5
total_writes_processed:4
io_threaded_reads_processed:0
io_threaded_writes_processed:0

# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:2865fe953ec34e04edf07b499a9a3c4fdd8cd18e
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

# CPU
used_cpu_sys:0.085222
used_cpu_user:0.100261
used_cpu_sys_children:0.000000
used_cpu_user_children:0.005942
used_cpu_sys_main_thread:0.082016
used_cpu_user_main_thread:0.096490

# Modules
module:name=interval_set,ver=1,api=1,filters=0,usedby=[],using=[],options=[]

# Commandstats
cmdstat_info:calls=1,usec=163,usec_per_call=163.00,rejected_calls=0,failed_calls=0
cmdstat_iset.add:calls=1,usec=132,usec_per_call=132.00,rejected_calls=0,failed_calls=0
cmdstat_flushdb:calls=1,usec=24,usec_per_call=24.00,rejected_calls=0,failed_calls=0
cmdstat_command:calls=1,usec=1515,usec_per_call=1515.00,rejected_calls=0,failed_calls=0

# Errorstats

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=1,expires=0,avg_ttl=0

------ CLIENT LIST OUTPUT ------
id=3 addr=172.17.0.1:34060 laddr=172.17.0.2:6379 fd=8 name= age=20 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=14 qbuf-free=40940 argv-mem=4 obl=0 oll=0 omem=0 tot-mem=61452 events=r cmd=save user=default redir=-1

------ CURRENT CLIENT INFO ------
id=3 addr=172.17.0.1:34060 laddr=172.17.0.2:6379 fd=8 name= age=20 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=14 qbuf-free=40940 argv-mem=4 obl=0 oll=0 omem=0 tot-mem=61452 events=r cmd=save user=default redir=-1
argv[0]: 'save'

------ MODULES INFO OUTPUT ------

------ FAST MEMORY TEST ------
1:M 01 Jan 2022 07:02:35.473 # Bio thread for job type #0 terminated
1:M 01 Jan 2022 07:02:35.473 # Bio thread for job type #1 terminated
1:M 01 Jan 2022 07:02:35.473 # Bio thread for job type #2 terminated
*** Preparing to test memory region 55ea97f77000 (2281472 bytes)
*** Preparing to test memory region 55ea986a6000 (270336 bytes)
*** Preparing to test memory region 7fbb74dfd000 (8388608 bytes)
*** Preparing to test memory region 7fbb755fe000 (8388608 bytes)
*** Preparing to test memory region 7fbb75dff000 (8388608 bytes)
*** Preparing to test memory region 7fbb76600000 (8388608 bytes)
*** Preparing to test memory region 7fbb76e00000 (8388608 bytes)
*** Preparing to test memory region 7fbb77765000 (16384 bytes)
*** Preparing to test memory region 7fbb77926000 (24576 bytes)
*** Preparing to test memory region 7fbb77949000 (16384 bytes)
*** Preparing to test memory region 7fbb77c32000 (16384 bytes)
*** Preparing to test memory region 7fbb77e5a000 (8192 bytes)
*** Preparing to test memory region 7fbb77e87000 (4096 bytes)
.O.O.O.O.O.O.O.O.O.O.O.O.O
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.

=== REDIS BUG REPORT END. Make sure to include from START to END. ===

       Please report the crash by opening an issue on github:

           http://github.com/redis/redis/issues

  Suspect RAM error? Use redis-server --test-memory to verify it.
stopforumspam commented 2 years ago

and then built on ubuntu 20.04 using the dockerfile in the repo tested with flsuhb/iadd/set as above

$sudo docker build .
Sending build context to Docker daemon  461.6MB
Step 1/12 : FROM rust:latest as builder
latest: Pulling from library/rust
0e29546d541c: Pull complete 
9b829c73b52b: Pull complete 
cb5b7ae36172: Pull complete 
6494e4811622: Pull complete 
6f9f74896dfa: Pull complete 
4afc806e0919: Pull complete 
Digest: sha256:d83bf5ea7b4c3d18c2f46d5f3d288bfca085c3e7ac57822e3b8e5a1ad22ccc1a
Status: Downloaded newer image for rust:latest
 ---> 4db2e2d14f99
Step 2/12 : ENV LIBDIR /usr/lib/redis/modules
 ---> Running in 8f5d10510553
Removing intermediate container 8f5d10510553
 ---> 6398fa739ebc
Step 3/12 : ADD . /RIS
 ---> a909d89abf95
Step 4/12 : WORKDIR /RIS
 ---> Running in abe3324fa222
Removing intermediate container abe3324fa222
 ---> b9a0a0a2fa95
Step 5/12 : RUN apt-get update -y && apt-get install git wget clang cmake -y
 ---> Running in 0c93d9d5de35
Get:1 http://security.debian.org/debian-security bullseye-security InRelease [44.1 kB]
Get:2 http://deb.debian.org/debian bullseye InRelease [116 kB]
Get:3 http://security.debian.org/debian-security bullseye-security/main amd64 Packages [103 kB]
Get:4 http://deb.debian.org/debian bullseye-updates InRelease [39.4 kB]
Get:5 http://deb.debian.org/debian bullseye/main amd64 Packages [8183 kB]
Get:6 http://deb.debian.org/debian bullseye-updates/main amd64 Packages [2592 B]
Fetched 8488 kB in 9s (901 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
git is already the newest version (1:2.30.2-1).
wget is already the newest version (1.21-1+deb11u1).
The following additional packages will be installed:
  binfmt-support clang-11 cmake-data lib32gcc-s1 lib32stdc++6 libarchive13
  libc6-i386 libclang-common-11-dev libclang-cpp11 libclang1-11 libgc1
  libjsoncpp24 libllvm11 libobjc-10-dev libobjc4 libomp-11-dev libomp5-11
  libpfm4 libpipeline1 librhash0 libuv1 libz3-4 libz3-dev llvm-11 llvm-11-dev
  llvm-11-runtime llvm-11-tools python3-pkg-resources python3-pygments
  python3-yaml
Suggested packages:
  clang-11-doc cmake-doc ninja-build lrzip libomp-11-doc llvm-11-doc
  python3-setuptools python-pygments-doc ttf-bitstream-vera
The following NEW packages will be installed:
  binfmt-support clang clang-11 cmake cmake-data lib32gcc-s1 lib32stdc++6
  libarchive13 libc6-i386 libclang-common-11-dev libclang-cpp11 libclang1-11
  libgc1 libjsoncpp24 libllvm11 libobjc-10-dev libobjc4 libomp-11-dev
  libomp5-11 libpfm4 libpipeline1 librhash0 libuv1 libz3-4 libz3-dev llvm-11
  llvm-11-dev llvm-11-runtime llvm-11-tools python3-pkg-resources
  python3-pygments python3-yaml
0 upgraded, 32 newly installed, 0 to remove and 0 not upgraded.
Need to get 95.6 MB of archives.
After this operation, 559 MB of additional disk space will be used.
Get:1 http://deb.debian.org/debian bullseye/main amd64 libpipeline1 amd64 1.5.3-1 [34.3 kB]
Get:2 http://deb.debian.org/debian bullseye/main amd64 binfmt-support amd64 2.2.1-1 [66.1 kB]
Get:3 http://deb.debian.org/debian bullseye/main amd64 libz3-4 amd64 4.8.10-1 [6949 kB]
Get:4 http://deb.debian.org/debian bullseye/main amd64 libllvm11 amd64 1:11.0.1-2 [17.9 MB]
Get:5 http://deb.debian.org/debian bullseye/main amd64 libclang-cpp11 amd64 1:11.0.1-2 [9632 kB]
Get:6 http://deb.debian.org/debian bullseye/main amd64 libgc1 amd64 1:8.0.4-3 [239 kB]
Get:7 http://deb.debian.org/debian bullseye/main amd64 libobjc4 amd64 10.2.1-6 [41.8 kB]
Get:8 http://deb.debian.org/debian bullseye/main amd64 libobjc-10-dev amd64 10.2.1-6 [170 kB]
Get:9 http://deb.debian.org/debian bullseye/main amd64 libc6-i386 amd64 2.31-13+deb11u2 [2616 kB]
Get:10 http://deb.debian.org/debian bullseye/main amd64 lib32gcc-s1 amd64 10.2.1-6 [49.4 kB]
Get:11 http://deb.debian.org/debian bullseye/main amd64 lib32stdc++6 amd64 10.2.1-6 [510 kB]
Get:12 http://deb.debian.org/debian bullseye/main amd64 libclang-common-11-dev amd64 1:11.0.1-2 [4922 kB]
Get:13 http://deb.debian.org/debian bullseye/main amd64 libclang1-11 amd64 1:11.0.1-2 [5521 kB]
Get:14 http://deb.debian.org/debian bullseye/main amd64 clang-11 amd64 1:11.0.1-2 [109 kB]
Get:15 http://deb.debian.org/debian bullseye/main amd64 clang amd64 1:11.0-51+nmu5 [8872 B]
Get:16 http://deb.debian.org/debian bullseye/main amd64 cmake-data all 3.18.4-2+deb11u1 [1725 kB]
Get:17 http://deb.debian.org/debian bullseye/main amd64 libarchive13 amd64 3.4.3-2+b1 [343 kB]
Get:18 http://deb.debian.org/debian bullseye/main amd64 libjsoncpp24 amd64 1.9.4-4 [78.9 kB]
Get:19 http://deb.debian.org/debian bullseye/main amd64 librhash0 amd64 1.4.1-2 [129 kB]
Get:20 http://deb.debian.org/debian bullseye/main amd64 libuv1 amd64 1.40.0-2 [132 kB]
Get:21 http://deb.debian.org/debian bullseye/main amd64 cmake amd64 3.18.4-2+deb11u1 [5593 kB]
Get:22 http://deb.debian.org/debian bullseye/main amd64 libomp5-11 amd64 1:11.0.1-2 [346 kB]
Get:23 http://deb.debian.org/debian bullseye/main amd64 libomp-11-dev amd64 1:11.0.1-2 [94.1 kB]
Get:24 http://deb.debian.org/debian bullseye/main amd64 libpfm4 amd64 4.11.1+git32-gd0b85fb-1 [286 kB]
Get:25 http://deb.debian.org/debian bullseye/main amd64 libz3-dev amd64 4.8.10-1 [90.8 kB]
Get:26 http://deb.debian.org/debian bullseye/main amd64 llvm-11-runtime amd64 1:11.0.1-2 [212 kB]
Get:27 http://deb.debian.org/debian bullseye/main amd64 llvm-11 amd64 1:11.0.1-2 [8463 kB]
Get:28 http://deb.debian.org/debian bullseye/main amd64 python3-pkg-resources all 52.0.0-4 [190 kB]
Get:29 http://deb.debian.org/debian bullseye/main amd64 python3-pygments all 2.7.1+dfsg-2.1 [657 kB]
Get:30 http://deb.debian.org/debian bullseye/main amd64 python3-yaml amd64 5.3.1-5 [138 kB]
Get:31 http://deb.debian.org/debian bullseye/main amd64 llvm-11-tools amd64 1:11.0.1-2 [373 kB]
Get:32 http://deb.debian.org/debian bullseye/main amd64 llvm-11-dev amd64 1:11.0.1-2 [28.0 MB]
debconf: delaying package configuration, since apt-utils is not installed
Fetched 95.6 MB in 6s (15.5 MB/s)
Selecting previously unselected package libpipeline1:amd64.
(Reading database ... 22741 files and directories currently installed.)
Preparing to unpack .../00-libpipeline1_1.5.3-1_amd64.deb ...
Unpacking libpipeline1:amd64 (1.5.3-1) ...
Selecting previously unselected package binfmt-support.
Preparing to unpack .../01-binfmt-support_2.2.1-1_amd64.deb ...
Unpacking binfmt-support (2.2.1-1) ...
Selecting previously unselected package libz3-4:amd64.
Preparing to unpack .../02-libz3-4_4.8.10-1_amd64.deb ...
Unpacking libz3-4:amd64 (4.8.10-1) ...
Selecting previously unselected package libllvm11:amd64.
Preparing to unpack .../03-libllvm11_1%3a11.0.1-2_amd64.deb ...
Unpacking libllvm11:amd64 (1:11.0.1-2) ...
Selecting previously unselected package libclang-cpp11.
Preparing to unpack .../04-libclang-cpp11_1%3a11.0.1-2_amd64.deb ...
Unpacking libclang-cpp11 (1:11.0.1-2) ...
Selecting previously unselected package libgc1:amd64.
Preparing to unpack .../05-libgc1_1%3a8.0.4-3_amd64.deb ...
Unpacking libgc1:amd64 (1:8.0.4-3) ...
Selecting previously unselected package libobjc4:amd64.
Preparing to unpack .../06-libobjc4_10.2.1-6_amd64.deb ...
Unpacking libobjc4:amd64 (10.2.1-6) ...
Selecting previously unselected package libobjc-10-dev:amd64.
Preparing to unpack .../07-libobjc-10-dev_10.2.1-6_amd64.deb ...
Unpacking libobjc-10-dev:amd64 (10.2.1-6) ...
Selecting previously unselected package libc6-i386.
Preparing to unpack .../08-libc6-i386_2.31-13+deb11u2_amd64.deb ...
Unpacking libc6-i386 (2.31-13+deb11u2) ...
Selecting previously unselected package lib32gcc-s1.
Preparing to unpack .../09-lib32gcc-s1_10.2.1-6_amd64.deb ...
Unpacking lib32gcc-s1 (10.2.1-6) ...
Selecting previously unselected package lib32stdc++6.
Preparing to unpack .../10-lib32stdc++6_10.2.1-6_amd64.deb ...
Unpacking lib32stdc++6 (10.2.1-6) ...
Selecting previously unselected package libclang-common-11-dev.
Preparing to unpack .../11-libclang-common-11-dev_1%3a11.0.1-2_amd64.deb ...
Unpacking libclang-common-11-dev (1:11.0.1-2) ...
Selecting previously unselected package libclang1-11.
Preparing to unpack .../12-libclang1-11_1%3a11.0.1-2_amd64.deb ...
Unpacking libclang1-11 (1:11.0.1-2) ...
Selecting previously unselected package clang-11.
Preparing to unpack .../13-clang-11_1%3a11.0.1-2_amd64.deb ...
Unpacking clang-11 (1:11.0.1-2) ...
Selecting previously unselected package clang.
Preparing to unpack .../14-clang_1%3a11.0-51+nmu5_amd64.deb ...
Unpacking clang (1:11.0-51+nmu5) ...
Selecting previously unselected package cmake-data.
Preparing to unpack .../15-cmake-data_3.18.4-2+deb11u1_all.deb ...
Unpacking cmake-data (3.18.4-2+deb11u1) ...
Selecting previously unselected package libarchive13:amd64.
Preparing to unpack .../16-libarchive13_3.4.3-2+b1_amd64.deb ...
Unpacking libarchive13:amd64 (3.4.3-2+b1) ...
Selecting previously unselected package libjsoncpp24:amd64.
Preparing to unpack .../17-libjsoncpp24_1.9.4-4_amd64.deb ...
Unpacking libjsoncpp24:amd64 (1.9.4-4) ...
Selecting previously unselected package librhash0:amd64.
Preparing to unpack .../18-librhash0_1.4.1-2_amd64.deb ...
Unpacking librhash0:amd64 (1.4.1-2) ...
Selecting previously unselected package libuv1:amd64.
Preparing to unpack .../19-libuv1_1.40.0-2_amd64.deb ...
Unpacking libuv1:amd64 (1.40.0-2) ...
Selecting previously unselected package cmake.
Preparing to unpack .../20-cmake_3.18.4-2+deb11u1_amd64.deb ...
Unpacking cmake (3.18.4-2+deb11u1) ...
Selecting previously unselected package libomp5-11:amd64.
Preparing to unpack .../21-libomp5-11_1%3a11.0.1-2_amd64.deb ...
Unpacking libomp5-11:amd64 (1:11.0.1-2) ...
Selecting previously unselected package libomp-11-dev.
Preparing to unpack .../22-libomp-11-dev_1%3a11.0.1-2_amd64.deb ...
Unpacking libomp-11-dev (1:11.0.1-2) ...
Selecting previously unselected package libpfm4:amd64.
Preparing to unpack .../23-libpfm4_4.11.1+git32-gd0b85fb-1_amd64.deb ...
Unpacking libpfm4:amd64 (4.11.1+git32-gd0b85fb-1) ...
Selecting previously unselected package libz3-dev:amd64.
Preparing to unpack .../24-libz3-dev_4.8.10-1_amd64.deb ...
Unpacking libz3-dev:amd64 (4.8.10-1) ...
Selecting previously unselected package llvm-11-runtime.
Preparing to unpack .../25-llvm-11-runtime_1%3a11.0.1-2_amd64.deb ...
Unpacking llvm-11-runtime (1:11.0.1-2) ...
Selecting previously unselected package llvm-11.
Preparing to unpack .../26-llvm-11_1%3a11.0.1-2_amd64.deb ...
Unpacking llvm-11 (1:11.0.1-2) ...
Selecting previously unselected package python3-pkg-resources.
Preparing to unpack .../27-python3-pkg-resources_52.0.0-4_all.deb ...
Unpacking python3-pkg-resources (52.0.0-4) ...
Selecting previously unselected package python3-pygments.
Preparing to unpack .../28-python3-pygments_2.7.1+dfsg-2.1_all.deb ...
Unpacking python3-pygments (2.7.1+dfsg-2.1) ...
Selecting previously unselected package python3-yaml.
Preparing to unpack .../29-python3-yaml_5.3.1-5_amd64.deb ...
Unpacking python3-yaml (5.3.1-5) ...
Selecting previously unselected package llvm-11-tools.
Preparing to unpack .../30-llvm-11-tools_1%3a11.0.1-2_amd64.deb ...
Unpacking llvm-11-tools (1:11.0.1-2) ...
Selecting previously unselected package llvm-11-dev.
Preparing to unpack .../31-llvm-11-dev_1%3a11.0.1-2_amd64.deb ...
Unpacking llvm-11-dev (1:11.0.1-2) ...
Setting up python3-pkg-resources (52.0.0-4) ...
Setting up libpipeline1:amd64 (1.5.3-1) ...
Setting up libarchive13:amd64 (3.4.3-2+b1) ...
Setting up python3-yaml (5.3.1-5) ...
Setting up python3-pygments (2.7.1+dfsg-2.1) ...
Setting up libz3-4:amd64 (4.8.10-1) ...
Setting up libuv1:amd64 (1.40.0-2) ...
Setting up libpfm4:amd64 (4.11.1+git32-gd0b85fb-1) ...
Setting up libllvm11:amd64 (1:11.0.1-2) ...
Setting up libclang1-11 (1:11.0.1-2) ...
Setting up binfmt-support (2.2.1-1) ...
invoke-rc.d: could not determine current runlevel
invoke-rc.d: policy-rc.d denied execution of start.
Setting up libgc1:amd64 (1:8.0.4-3) ...
Setting up libomp5-11:amd64 (1:11.0.1-2) ...
Setting up libjsoncpp24:amd64 (1.9.4-4) ...
Setting up librhash0:amd64 (1.4.1-2) ...
Setting up libc6-i386 (2.31-13+deb11u2) ...
Setting up cmake-data (3.18.4-2+deb11u1) ...
Setting up libz3-dev:amd64 (4.8.10-1) ...
Setting up llvm-11-runtime (1:11.0.1-2) ...
Setting up libomp-11-dev (1:11.0.1-2) ...
Setting up llvm-11-tools (1:11.0.1-2) ...
Setting up llvm-11 (1:11.0.1-2) ...
Setting up libobjc4:amd64 (10.2.1-6) ...
Setting up libclang-cpp11 (1:11.0.1-2) ...
Setting up lib32gcc-s1 (10.2.1-6) ...
Setting up lib32stdc++6 (10.2.1-6) ...
Setting up libobjc-10-dev:amd64 (10.2.1-6) ...
Setting up libclang-common-11-dev (1:11.0.1-2) ...
Setting up cmake (3.18.4-2+deb11u1) ...
Setting up clang-11 (1:11.0.1-2) ...
Setting up llvm-11-dev (1:11.0.1-2) ...
Setting up clang (1:11.0-51+nmu5) ...
Processing triggers for libc-bin (2.31-13+deb11u2) ...
Removing intermediate container 0c93d9d5de35
 ---> 097bc219d41f
Step 6/12 : RUN set -ex ;    cargo build --release ;    mv target/release/libintervalsets.so target/release/ris.so
 ---> Running in f700f19728ab
+ cargo build --release
    Updating crates.io index
 Downloading crates ...
  Downloaded atty v0.2.14
  Downloaded libloading v0.6.7
  Downloaded cexpr v0.4.0
  Downloaded rustc-hash v1.1.0
  Downloaded proc-macro2 v1.0.24
  Downloaded termcolor v1.1.2
  Downloaded autocfg v1.0.1
  Downloaded vec_map v0.8.2
  Downloaded cfg-if v1.0.0
  Downloaded unicode-segmentation v1.7.1
  Downloaded aho-corasick v0.7.15
  Downloaded nom v5.1.2
  Downloaded memchr v2.3.4
  Downloaded which v3.1.1
  Downloaded shlex v0.1.1
  Downloaded thread_local v1.1.3
  Downloaded unicode-xid v0.0.4
  Downloaded textwrap v0.11.0
  Downloaded synom v0.11.3
  Downloaded regex v1.4.3
  Downloaded peeking_take_while v0.1.2
  Downloaded num-traits v0.2.14
  Downloaded log v0.4.14
  Downloaded glob v0.3.0
  Downloaded quote v1.0.9
  Downloaded lazycell v1.3.0
  Downloaded once_cell v1.5.2
  Downloaded clap v2.33.3
  Downloaded env_logger v0.8.3
  Downloaded libc v0.2.86
  Downloaded version_check v0.9.2
  Downloaded unicode-xid v0.2.1
  Downloaded strum_macros v0.19.4
  Downloaded cc v1.0.66
  Downloaded strsim v0.8.0
  Downloaded unicode-width v0.1.8
  Downloaded humantime v2.1.0
  Downloaded regex-syntax v0.6.22
  Downloaded heck v0.3.2
  Downloaded syn v1.0.60
  Downloaded quote v0.3.15
  Downloaded lazy_static v1.4.0
  Downloaded syn v0.11.11
  Downloaded bindgen v0.57.0
  Downloaded bitflags v1.2.1
  Downloaded ansi_term v0.11.0
  Downloaded num-traits v0.1.43
  Downloaded enum-primitive-derive v0.1.2
  Downloaded redis-module v0.12.0
  Downloaded clang-sys v1.1.0
   Compiling libc v0.2.86
   Compiling memchr v2.3.4
   Compiling bitflags v1.2.1
   Compiling proc-macro2 v1.0.24
   Compiling cfg-if v1.0.0
   Compiling autocfg v1.0.1
   Compiling unicode-xid v0.2.1
   Compiling version_check v0.9.2
   Compiling glob v0.3.0
   Compiling once_cell v1.5.2
   Compiling log v0.4.14
   Compiling unicode-width v0.1.8
   Compiling regex-syntax v0.6.22
   Compiling humantime v2.1.0
   Compiling termcolor v1.1.2
   Compiling vec_map v0.8.2
   Compiling strsim v0.8.0
   Compiling ansi_term v0.11.0
   Compiling bindgen v0.57.0
   Compiling lazy_static v1.4.0
   Compiling rustc-hash v1.1.0
   Compiling unicode-xid v0.0.4
   Compiling peeking_take_while v0.1.2
   Compiling syn v1.0.60
   Compiling shlex v0.1.1
   Compiling lazycell v1.3.0
   Compiling cc v1.0.66
   Compiling unicode-segmentation v1.7.1
   Compiling quote v0.3.15
   Compiling libloading v0.6.7
   Compiling thread_local v1.1.3
   Compiling textwrap v0.11.0
   Compiling nom v5.1.2
   Compiling num-traits v0.2.14
   Compiling clang-sys v1.1.0
   Compiling synom v0.11.3
   Compiling heck v0.3.2
   Compiling syn v0.11.11
   Compiling aho-corasick v0.7.15
   Compiling quote v1.0.9
   Compiling atty v0.2.14
   Compiling which v3.1.1
   Compiling clap v2.33.3
   Compiling regex v1.4.3
   Compiling num-traits v0.1.43
   Compiling enum-primitive-derive v0.1.2
   Compiling cexpr v0.4.0
   Compiling env_logger v0.8.3
   Compiling strum_macros v0.19.4
   Compiling redis-module v0.12.0
   Compiling intervalsets v0.0.1 (/RIS)
    Finished release [optimized] target(s) in 1m 10s
+ mv target/release/libintervalsets.so target/release/ris.so
Removing intermediate container f700f19728ab
 ---> d185d458b59b
Step 7/12 : FROM redis:latest
latest: Pulling from library/redis
a2abf6c4d29d: Pull complete 
c7a4e4382001: Pull complete 
4044b9ba67c9: Pull complete 
c8388a79482f: Pull complete 
413c8bb60be2: Pull complete 
1abfd3011519: Pull complete 
Digest: sha256:db485f2e245b5b3329fdc7eff4eb00f913e09d8feb9ca720788059fdc2ed8339
Status: Downloaded newer image for redis:latest
 ---> 7614ae9453d1
Step 8/12 : ENV LIBDIR /usr/lib/redis/modules
 ---> Running in 8f7ff6687e9f
Removing intermediate container 8f7ff6687e9f
 ---> 554274318a01
Step 9/12 : WORKDIR /data
 ---> Running in 4192dcaeade7
Removing intermediate container 4192dcaeade7
 ---> 478db87857e7
Step 10/12 : RUN mkdir -p "$LIBDIR"
 ---> Running in c832124a36c1
Removing intermediate container c832124a36c1
 ---> 94f9581ef3fb
Step 11/12 : COPY --from=builder /RIS/target/release/ris.so "$LIBDIR"
 ---> 6723e31d4c3c
Step 12/12 : CMD ["redis-server", "--loadmodule", "/usr/lib/redis/modules/ris.so"]
 ---> Running in 719e592d25eb
Removing intermediate container 719e592d25eb
 ---> eaa0f85d947d
Successfully built eaa0f85d947d

and 

sudo docker run -p 6379:6379 eaa0f85d947d
1:C 01 Jan 2022 07:13:01.897 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 01 Jan 2022 07:13:01.897 # Redis version=6.2.6, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 01 Jan 2022 07:13:01.897 # Configuration loaded
1:M 01 Jan 2022 07:13:01.898 * monotonic clock: POSIX clock_gettime
1:M 01 Jan 2022 07:13:01.899 * Running mode=standalone, port=6379.
1:M 01 Jan 2022 07:13:01.899 # Server initialized
1:M 01 Jan 2022 07:13:01.900 * <interval_set> Created new data type 'IntervlSt'
1:M 01 Jan 2022 07:13:01.900 * Module 'interval_set' loaded from /usr/lib/redis/modules/ris.so
1:M 01 Jan 2022 07:13:01.900 * Ready to accept connections
[iset.add] Adding a new key 'test'

=== REDIS BUG REPORT START: Cut & paste starting from here ===
1:M 01 Jan 2022 07:13:21.541 # Redis 6.2.6 crashed by signal: 11, si_code: 1
1:M 01 Jan 2022 07:13:21.541 # Accessing address: (nil)

------ STACK TRACE ------

Backtrace:
/lib/x86_64-linux-gnu/libpthread.so.0(+0x14140)[0x7f6555131140]

------ REGISTERS ------
1:M 01 Jan 2022 07:13:21.541 # 
RAX:0000000000000009 RBX:00007f6554a22080
RCX:00b452f9ae5e7b22 RDX:00007fff7cdfdb40
RDI:00007fff7cdfd870 RSI:00007f6554b3fb00
RBP:00007fff7cdfdba0 RSP:00007fff7cdfd848
R8 :0000000000000008 R9 :00007fff7cdfd800
R10:00000000000000f4 R11:00007f65550ca3c0
R12:00007f6554a7be20 R13:00007fff7cdfdb40
R14:0000000000000000 R15:0000000000000000
RIP:0000000000000000 EFL:0000000000010217
CSGSFS:002b000000000033
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd857) -> 0000000000000000
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd856) -> 0000000000000000
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd855) -> 0000000000000008
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd854) -> 0000007700000063
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd853) -> 0000000000000001
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd852) -> 00007fff7cdfdb40
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd851) -> 0000000000000000
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd850) -> 0000000000000000
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd84f) -> 00007f6554a22080
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd84e) -> 00007fff7cdfdba0
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd84d) -> 0000000000000009
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd84c) -> 0000000000001000
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd84b) -> 0000000000000000
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd84a) -> 00007fff7cdfdb40
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd849) -> 00000000000000ff
1:M 01 Jan 2022 07:13:21.541 # (00007fff7cdfd848) -> 00005571371e3c50

------ INFO OUTPUT ------
# Server
redis_version:6.2.6
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:b61f37314a089f19
redis_mode:standalone
os:Linux 5.4.0-90-generic x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:10.2.1
process_id:1
process_supervised:no
run_id:c027842955e4d752b3939dbabaa17417def1a085
tcp_port:6379
server_time_usec:1641021201541355
uptime_in_seconds:20
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:13631249
executable:/data/redis-server
config_file:
io_threads_active:0

# Clients
connected_clients:1
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:48
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0

# Memory
used_memory:874992
used_memory_human:854.48K
used_memory_rss:8650752
used_memory_rss_human:8.25M
used_memory_peak:932456
used_memory_peak_human:910.60K
used_memory_peak_perc:93.84%
used_memory_overhead:830504
used_memory_startup:809904
used_memory_dataset:44488
used_memory_dataset_perc:68.35%
allocator_allocated:1207200
allocator_active:1449984
allocator_resident:4308992
total_system_memory:16790593536
total_system_memory_human:15.64G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.20
allocator_frag_bytes:242784
allocator_rss_ratio:2.97
allocator_rss_bytes:2859008
rss_overhead_ratio:2.01
rss_overhead_bytes:4341760
mem_fragmentation_ratio:10.39
mem_fragmentation_bytes:7818544
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_clients_slaves:0
mem_clients_normal:20528
mem_aof_buffer:0
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0

# Persistence
loading:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1641021181
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

# Stats
total_connections_received:1
total_commands_processed:5
instantaneous_ops_per_sec:0
total_net_input_bytes:129
total_net_output_bytes:24752
instantaneous_input_kbps:0.03
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:0
dump_payload_sanitizations:0
total_reads_processed:6
total_writes_processed:5
io_threaded_reads_processed:0
io_threaded_writes_processed:0

# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:28094f2dba2249d09b049defc281f91dad5e3013
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

# CPU
used_cpu_sys:0.051959
used_cpu_user:0.064949
used_cpu_sys_children:0.004374
used_cpu_user_children:0.002056
used_cpu_sys_main_thread:0.049151
used_cpu_user_main_thread:0.061439

# Modules
module:name=interval_set,ver=1,api=1,filters=0,usedby=[],using=[],options=[]

# Commandstats
cmdstat_info:calls=1,usec=192,usec_per_call=192.00,rejected_calls=0,failed_calls=0
cmdstat_iset.add:calls=1,usec=70,usec_per_call=70.00,rejected_calls=0,failed_calls=0
cmdstat_command:calls=1,usec=758,usec_per_call=758.00,rejected_calls=0,failed_calls=0
cmdstat_flushdb:calls=2,usec=36,usec_per_call=18.00,rejected_calls=0,failed_calls=0

# Errorstats

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=1,expires=0,avg_ttl=0

------ CLIENT LIST OUTPUT ------
id=3 addr=172.17.0.1:34164 laddr=172.17.0.2:6379 fd=8 name= age=13 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=14 qbuf-free=40940 argv-mem=4 obl=0 oll=0 omem=0 tot-mem=61452 events=r cmd=save user=default redir=-1

------ CURRENT CLIENT INFO ------
id=3 addr=172.17.0.1:34164 laddr=172.17.0.2:6379 fd=8 name= age=13 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=14 qbuf-free=40940 argv-mem=4 obl=0 oll=0 omem=0 tot-mem=61452 events=r cmd=save user=default redir=-1
argv[0]: 'save'

------ MODULES INFO OUTPUT ------

------ FAST MEMORY TEST ------
1:M 01 Jan 2022 07:13:21.542 # Bio thread for job type #0 terminated
1:M 01 Jan 2022 07:13:21.542 # Bio thread for job type #1 terminated
1:M 01 Jan 2022 07:13:21.542 # Bio thread for job type #2 terminated
*** Preparing to test memory region 55713737c000 (2277376 bytes)
*** Preparing to test memory region 55713918f000 (270336 bytes)
*** Preparing to test memory region 7f65525fd000 (8388608 bytes)
*** Preparing to test memory region 7f6552dfe000 (8388608 bytes)
*** Preparing to test memory region 7f65535ff000 (8388608 bytes)
*** Preparing to test memory region 7f6553e00000 (8388608 bytes)
*** Preparing to test memory region 7f6554600000 (8388608 bytes)
*** Preparing to test memory region 7f6554f52000 (24576 bytes)
*** Preparing to test memory region 7f6555119000 (16384 bytes)
*** Preparing to test memory region 7f655513b000 (16384 bytes)
*** Preparing to test memory region 7f655542f000 (16384 bytes)
*** Preparing to test memory region 7f6555610000 (8192 bytes)
*** Preparing to test memory region 7f6555640000 (4096 bytes)
.O.O.O.O.O.O.O.O.O.O.O.O.O
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.

=== REDIS BUG REPORT END. Make sure to include from START to END. ===

       Please report the crash by opening an issue on github:

           http://github.com/redis/redis/issues

  Suspect RAM error? Use redis-server --test-memory to verify it.
stopforumspam commented 2 years ago

and then in gdb

$ gdb  --args ./redis-server --loadmodule /home/coder/code/redis-interval-sets/target/debug/libintervalsets.so
GNU gdb (Ubuntu 9.2-0ubuntu1~20.04) 9.2
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./redis-server...
(gdb) run
Starting program: /home/coder/code/redis/src/redis-server --loadmodule /home/coder/code/redis-interval-sets/target/debug/libintervalsets.so
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
1128980:C 01 Jan 2022 07:17:03.092 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1128980:C 01 Jan 2022 07:17:03.093 # Redis version=255.255.255, bits=64, commit=39ced1af, modified=0, pid=1128980, just started
1128980:C 01 Jan 2022 07:17:03.093 # Configuration loaded
1128980:M 01 Jan 2022 07:17:03.093 * monotonic clock: POSIX clock_gettime
                _._                                                  
           _.-``__ ''-._                                             
      _.-``    `.  `_.  ''-._           Redis 255.255.255 (39ced1af/0) 64 bit
  .-`` .-```.  ```\/    _.,_ ''-._                                  
 (    '      ,       .-`  | `,    )     Running in standalone mode
 |`-._`-...-` __...-.``-._|'` _.-'|     Port: 6379
 |    `-._   `._    /     _.-'    |     PID: 1128980
  `-._    `-._  `-./  _.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |           https://redis.io       
  `-._    `-._`-.__.-'_.-'    _.-'                                   
 |`-._`-._    `-.__.-'    _.-'_.-'|                                  
 |    `-._`-._        _.-'_.-'    |                                  
  `-._    `-._`-.__.-'_.-'    _.-'                                   
      `-._    `-.__.-'    _.-'                                       
          `-._        _.-'                                           
              `-.__.-'                                               

1128980:M 01 Jan 2022 07:17:03.095 # Server initialized
1128980:M 01 Jan 2022 07:17:03.190 * <interval_set> Created new data type 'IntervlSt'
1128980:M 01 Jan 2022 07:17:03.190 * Module 'interval_set' loaded from /home/coder/code/redis-interval-sets/target/debug/libintervalsets.so
[New Thread 0x7ffff6a8f700 (LWP 1128984)]
[New Thread 0x7ffff628e700 (LWP 1128985)]
[New Thread 0x7ffff5a8d700 (LWP 1128986)]
[New Thread 0x7ffff528c700 (LWP 1128987)]
1128980:M 01 Jan 2022 07:17:03.192 * Loading RDB produced by version 255.255.255
1128980:M 01 Jan 2022 07:17:03.192 * RDB age 1807 seconds
1128980:M 01 Jan 2022 07:17:03.192 * RDB memory usage when created 0.85 Mb
1128980:M 01 Jan 2022 07:17:03.192 * Done loading RDB, keys loaded: 0, keys expired: 0.
1128980:M 01 Jan 2022 07:17:03.192 * DB loaded from disk: 0.000 seconds
1128980:M 01 Jan 2022 07:17:03.192 * Ready to accept connections
1128980:M 01 Jan 2022 07:17:13.916 * DB saved on disk
[iset.add] Adding a new key 'test'

Thread 1 "redis-server" received signal SIGSEGV, Segmentation fault.
0x0000000000000000 in ?? ()
(gdb) bt
#0  0x0000000000000000 in ?? ()
#1  0x00005555555f1bfb in rdbSaveObject (rdb=0x7fffffffce30, o=<optimised out>, key=0x7fffffffcdb0, dbid=0) at rdb.c:1054
#2  0x00005555555f24ad in rdbSaveKeyValuePair (rdb=rdb@entry=0x7fffffffce30, key=key@entry=0x7fffffffcdb0, val=val@entry=0x7ffff74585a0, expiretime=<optimised out>, dbid=dbid@entry=0) at rdb.c:1118
#3  0x00005555555f2f63 in rdbSaveRio (rdb=rdb@entry=0x7fffffffce30, error=error@entry=0x7fffffffce2c, rdbflags=rdbflags@entry=0, rsi=rsi@entry=0x0) at rdb.c:1296
#4  0x00005555555f5dc3 in rdbSave (filename=0x7ffff740a010 "dump.rdb", rsi=0x0) at rdb.c:1404
#5  0x00005555555f7e4c in saveCommand (c=0x7ffff752a300) at rdb.c:3369
#6  0x00005555555bffaf in call (c=0x7ffff752a300, flags=15) at server.c:3006
#7  0x00005555555c104d in processCommand (c=c@entry=0x7ffff752a300) at server.c:3583
#8  0x00005555555d7790 in processCommandAndResetClient (c=c@entry=0x7ffff752a300) at networking.c:2156
#9  0x00005555555da1fd in processInputBuffer (c=c@entry=0x7ffff752a300) at networking.c:2251
#10 0x00005555555dd680 in readQueryFromClient (conn=<optimised out>) at networking.c:2363
#11 0x0000555555678cac in callHandler (handler=<optimised out>, conn=0x7ffff741e480) at connhelpers.h:79
#12 connSocketEventHandler (el=<optimised out>, fd=<optimised out>, clientData=0x7ffff741e480, mask=<optimised out>) at connection.c:295
#13 0x00005555555b7f7a in aeProcessEvents (eventLoop=eventLoop@entry=0x7ffff741f230, flags=flags@entry=27) at ae.c:428
#14 0x00005555555b833d in aeMain (eventLoop=0x7ffff741f230) at ae.c:488
#15 0x00005555555b4035 in main (argc=<optimised out>, argv=0x7fffffffe398) at server.c:6375

rdb.c line 1054

        /* Then write the module-specific representation + EOF marker. */
        mt->rdb_save(&io,mv->value);
danitseitlin commented 2 years ago

@stopforumspam I see, so the issue was save after setting an interval set. I will see what's up with it. Thanks

stopforumspam commented 2 years ago

Yes, it's saving. If you run redis empherically in memory with persistence then it runs without crashing

danitseitlin commented 2 years ago

@stopforumspam Does this also reproduce when using bgsave instead of save?

stopforumspam commented 2 years ago

sadly yes

$ redis-cli
127.0.0.1:6379> save
OK
127.0.0.1:6379> flushdb
OK
127.0.0.1:6379> save
OK
127.0.0.1:6379> flushdb
OK
127.0.0.1:6379> iset.add test ip 1 2
OK
127.0.0.1:6379> bgsave
Background saving started
127.0.0.1:6379>

And the output (run from docker container)

# sudo docker run -p 6379:6379 eaa0f85d947d
1:C 05 Jan 2022 04:29:33.842 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 05 Jan 2022 04:29:33.842 # Redis version=6.2.6, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 05 Jan 2022 04:29:33.842 # Configuration loaded
1:M 05 Jan 2022 04:29:33.843 * monotonic clock: POSIX clock_gettime
1:M 05 Jan 2022 04:29:33.844 * Running mode=standalone, port=6379.
1:M 05 Jan 2022 04:29:33.844 # Server initialized
1:M 05 Jan 2022 04:29:33.844 * <interval_set> Created new data type 'IntervlSt'
1:M 05 Jan 2022 04:29:33.844 * Module 'interval_set' loaded from /usr/lib/redis/modules/ris.so
1:M 05 Jan 2022 04:29:33.845 * Ready to accept connections
1:M 05 Jan 2022 04:29:52.544 * DB saved on disk
1:M 05 Jan 2022 04:29:57.130 * DB saved on disk
[iset.add] Adding a new key 'test'
1:M 05 Jan 2022 04:30:05.857 * Background saving started by pid 20

=== REDIS BUG REPORT START: Cut & paste starting from here ===
20:C 05 Jan 2022 04:30:05.858 # Redis 6.2.6 crashed by signal: 11, si_code: 1
20:C 05 Jan 2022 04:30:05.858 # Accessing address: (nil)

------ STACK TRACE ------

Backtrace:
/lib/x86_64-linux-gnu/libpthread.so.0(+0x14140)[0x7fe9b9a92140]

------ REGISTERS ------
20:C 05 Jan 2022 04:30:05.858 # 
RAX:0000000000000009 RBX:00007fe9b9422080
RCX:00b452f9ae5e7b22 RDX:00007ffdadb21da0
RDI:00007ffdadb21ad0 RSI:00007fe9b953fad0
RBP:00007ffdadb21e00 RSP:00007ffdadb21aa8
R8 :0000000000000008 R9 :00007ffdadb21a00
R10:00000000000000b0 R11:00007fe9b9a2b3c0
R12:00007fe9b947bf60 R13:00007ffdadb21da0
R14:0000000000000000 R15:0000000000000000
RIP:0000000000000000 EFL:0000000000010217
CSGSFS:002b000000000033
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21ab7) -> 00007ffdadb21d9f
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21ab6) -> 0000000000000000
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21ab5) -> 0000000000000008
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21ab4) -> 0000007700000063
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21ab3) -> 0000000000000001
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21ab2) -> 00007ffdadb21da0
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21ab1) -> 0000000000000000
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21ab0) -> 0000000000000000
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21aaf) -> 00007fe9b9422080
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21aae) -> 00007ffdadb21e00
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21aad) -> 0000000000000009
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21aac) -> 0000000000001000
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21aab) -> 0000000000000000
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21aaa) -> 00007ffdadb21da0
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21aa9) -> 00000000000000ff
20:C 05 Jan 2022 04:30:05.858 # (00007ffdadb21aa8) -> 00005650d7a92c50

------ INFO OUTPUT ------
# Server
redis_version:6.2.6
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:b61f37314a089f19
redis_mode:standalone
os:Linux 5.4.0-90-generic x86_64
arch_bits:64
multiplexing_api:epoll
atomicvar_api:atomic-builtin
gcc_version:10.2.1
process_id:20
process_supervised:no
run_id:1a5aaf080d0a80129e89f8ae7a8ebc8c79386112
tcp_port:6379
server_time_usec:1641357005857005
uptime_in_seconds:32
uptime_in_days:0
hz:10
configured_hz:10
lru_clock:13967053
executable:/data/redis-server
config_file:
io_threads_active:0

# Clients
connected_clients:1
cluster_connections:0
maxclients:10000
client_recent_max_input_buffer:48
client_recent_max_output_buffer:0
blocked_clients:0
tracking_clients:0
clients_in_timeout_table:0

# Memory
used_memory:875032
used_memory_human:854.52K
used_memory_rss:8581120
used_memory_rss_human:8.18M
used_memory_peak:932424
used_memory_peak_human:910.57K
used_memory_peak_perc:93.84%
used_memory_overhead:830504
used_memory_startup:809904
used_memory_dataset:44528
used_memory_dataset_perc:68.37%
allocator_allocated:1053968
allocator_active:1273856
allocator_resident:4132864
total_system_memory:16790593536
total_system_memory_human:15.64G
used_memory_lua:37888
used_memory_lua_human:37.00K
used_memory_scripts:0
used_memory_scripts_human:0B
number_of_cached_scripts:0
maxmemory:0
maxmemory_human:0B
maxmemory_policy:noeviction
allocator_frag_ratio:1.21
allocator_frag_bytes:219888
allocator_rss_ratio:3.24
allocator_rss_bytes:2859008
rss_overhead_ratio:2.08
rss_overhead_bytes:4448256
mem_fragmentation_ratio:10.31
mem_fragmentation_bytes:7748880
mem_not_counted_for_evict:0
mem_replication_backlog:0
mem_clients_slaves:0
mem_clients_normal:20528
mem_aof_buffer:0
mem_allocator:jemalloc-5.1.0
active_defrag_running:0
lazyfree_pending_objects:0
lazyfreed_objects:0

# Persistence
loading:0
current_cow_size:0
current_cow_size_age:0
current_fork_perc:0.00
current_save_keys_processed:0
current_save_keys_total:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1641356997
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:-1
rdb_current_bgsave_time_sec:-1
rdb_last_cow_size:0
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
aof_last_cow_size:0
module_fork_in_progress:0
module_fork_last_cow_size:0

# Stats
total_connections_received:1
total_commands_processed:6
instantaneous_ops_per_sec:0
total_net_input_bytes:160
total_net_output_bytes:20681
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
expired_stale_perc:0.00
expired_time_cap_reached_count:0
expire_cycle_cpu_milliseconds:0
evicted_keys:0
keyspace_hits:0
keyspace_misses:0
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:0
total_forks:0
migrate_cached_sockets:0
slave_expires_tracked_keys:0
active_defrag_hits:0
active_defrag_misses:0
active_defrag_key_hits:0
active_defrag_key_misses:0
tracking_total_keys:0
tracking_total_items:0
tracking_total_prefixes:0
unexpected_error_replies:0
total_error_replies:1
dump_payload_sanitizations:0
total_reads_processed:8
total_writes_processed:7
io_threaded_reads_processed:0
io_threaded_writes_processed:0

# Replication
role:master
connected_slaves:0
master_failover_state:no-failover
master_replid:7105d6026e74d9ad4302ebaa0b34b95b5870521f
master_replid2:0000000000000000000000000000000000000000
master_repl_offset:0
second_repl_offset:-1
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0

# CPU
used_cpu_sys:0.000000
used_cpu_user:0.001285
used_cpu_sys_children:0.000000
used_cpu_user_children:0.000000
used_cpu_sys_main_thread:0.000000
used_cpu_user_main_thread:0.001285

# Modules
module:name=interval_set,ver=1,api=1,filters=0,usedby=[],using=[],options=[]

# Commandstats
cmdstat_save:calls=2,usec=5554,usec_per_call=2777.00,rejected_calls=0,failed_calls=0
cmdstat_flushdb:calls=2,usec=75,usec_per_call=37.50,rejected_calls=0,failed_calls=0
cmdstat_iset.add:calls=1,usec=75,usec_per_call=75.00,rejected_calls=0,failed_calls=0
cmdstat_command:calls=1,usec=1522,usec_per_call=1522.00,rejected_calls=0,failed_calls=0

# Errorstats
errorstat_ERR:count=1

# Cluster
cluster_enabled:0

# Keyspace
db0:keys=1,expires=0,avg_ttl=0

------ CLIENT LIST OUTPUT ------
id=3 addr=172.17.0.1:34298 laddr=172.17.0.2:6379 fd=8 name= age=17 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=16 qbuf-free=40938 argv-mem=6 obl=0 oll=0 omem=0 tot-mem=61454 events=r cmd=bgsave user=default redir=-1

------ CURRENT CLIENT INFO ------
id=3 addr=172.17.0.1:34298 laddr=172.17.0.2:6379 fd=8 name= age=17 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=16 qbuf-free=40938 argv-mem=6 obl=0 oll=0 omem=0 tot-mem=61454 events=r cmd=bgsave user=default redir=-1
argv[0]: 'bgsave'

------ MODULES INFO OUTPUT ------

------ FAST MEMORY TEST ------
*** Preparing to test memory region 5650d7c2b000 (2277376 bytes)
*** Preparing to test memory region 5650d93e0000 (270336 bytes)
*** Preparing to test memory region 7fe9b6ffd000 (8388608 bytes)
*** Preparing to test memory region 7fe9b77fe000 (8388608 bytes)
*** Preparing to test memory region 7fe9b7fff000 (8388608 bytes)
*** Preparing to test memory region 7fe9b8800000 (8388608 bytes)
*** Preparing to test memory region 7fe9b9000000 (8388608 bytes)
*** Preparing to test memory region 7fe9b98b3000 (24576 bytes)
*** Preparing to test memory region 7fe9b9a7a000 (16384 bytes)
*** Preparing to test memory region 7fe9b9a9c000 (16384 bytes)
*** Preparing to test memory region 7fe9b9d90000 (16384 bytes)
*** Preparing to test memory region 7fe9b9f71000 (8192 bytes)
*** Preparing to test memory region 7fe9b9fa1000 (4096 bytes)
.O.O.O.O.O.O.O.O.O.O.O.O.O
Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.

=== REDIS BUG REPORT END. Make sure to include from START to END. ===

       Please report the crash by opening an issue on github:

           http://github.com/redis/redis/issues

  Suspect RAM error? Use redis-server --test-memory to verify it.

1:M 05 Jan 2022 04:30:06.498 # Background saving terminated by signal 11
stopforumspam commented 2 years ago

Could it be because you're setting data in Redis and when it saves, it has a null/None call pointer that should be returning a string?

eg

https://gitcode.net/mirrors/RedisJSON/RedisJSON/-/blob/53f8b291fc9adb4a399810c58847b95b76be852b/src/redisjson.rs#L510

and then again for loading

https://gitcode.net/mirrors/RedisJSON/RedisJSON/-/blob/53f8b291fc9adb4a399810c58847b95b76be852b/src/redisjson.rs#L478

I would add these but my Rust really is about zero

danitseitlin commented 2 years ago

@stopforumspam Hi, thanks for all the help! I also noticed this myself and I am working on support for rdb_save and rdb_load :) Stay tuned !

danitseitlin commented 2 years ago

@stopforumspam hi, wanted to sync you i finished fixing the bug. I just need to fix the tests and i will merge the fix.

stopforumspam commented 2 years ago

i've just started testing, so far so good :)

stopforumspam commented 2 years ago

so far its looking fantastic. memory usage vs my super memory-lite implementation using zsets for ip-to-country, ip-to-asn, and is-ip-in-blacklist is reduced by about 50% usage with your iset code, and with reduced overhead for maintaining them. when you have to run redis on a 512mb VPS, that really counts.

superb 👍

stopforumspam commented 2 years ago

I spoke too soon. a flush/import/save works, but if you shutdown redis and then load the rdb, then save it, it segfaults

$ redis-cli 127.0.0.1:6379> flushdb OK 127.0.0.1:6379> save OK 127.0.0.1:6379> iset.add iset_geo:ip 94177024 94177028 ro (error) Couldn't parse as integer 127.0.0.1:6379> iset.add iset_geo:ip to 94177024 94177028 OK 127.0.0.1:6379> iset.score iset_geo:ip 94177026 1) "to" 127.0.0.1:6379> save OK 127.0.0.1:6379> shutdown not connected> quit code@db:/code/redis-interval-sets$ redis-cli 127.0.0.1:6379> save OK 127.0.0.1:6379> iset.score iset_geo:ip 94177026 1) "to" 127.0.0.1:6379> quit code@db:/code/redis-interval-sets$ php test.php Executing...Done... code@db:/code/redis-interval-sets$ redis-cli 127.0.0.1:6379> iset.score iset_geo:ip 94177026 1) "us\x02\xef\xbf\xbd" 127.0.0.1:6379> save OK (11.62s) 127.0.0.1:6379> shutdown (11.70s) not connected> save (9.22s) not connected> (segfaults here)

and the redis dump

`$ src/redis-server redis.conf 215542:C 19 Feb 2022 06:10:05.881 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo 215542:C 19 Feb 2022 06:10:05.881 # Redis version=255.255.255, bits=64, commit=56fa48ff, modified=0, pid=215542, just started 215542:C 19 Feb 2022 06:10:05.881 # Configuration loaded 215542:M 19 Feb 2022 06:10:05.882 monotonic clock: POSIX clock_gettime 215542:M 19 Feb 2022 06:10:05.882 # Warning: Could not create server TCP listening socket ::1:6379: bind: Cannot assign requested address
215542:M 19 Feb 2022 06:10:05.883 # Server initialized 215542:M 19 Feb 2022 06:10:05.884 Created new data type 'IntervlSt' 215542:M 19 Feb 2022 06:10:05.884 Module 'interval_set' loaded from /home/code/code/redis-interval-sets/target/debug/deps/libintervalsets.so 215542:M 19 Feb 2022 06:10:05.884 The AOF directory appendonlydir doesn't exist 215542:M 19 Feb 2022 06:10:05.885 Loading RDB produced by version 255.255.255 215542:M 19 Feb 2022 06:10:05.885 RDB age 13 seconds 215542:M 19 Feb 2022 06:10:05.885 * RDB memory usage when created 24.90 Mb thread ' panicked at 'called Result::unwrap() on an Err value: ParseIntError { kind: InvalidDigit }', src/structs.rs:79:93 note: run with RUST_BACKTRACE=1 environment variable to display a backtrace fatal runtime error: failed to initiate panic, error 5 [removed] ------ MODULES INFO OUTPUT ------

interval_set_trace

interval_set_trace: 0: redis_module::base_info_func at /home/code/.cargo/registry/src/github.com-1ecc6299db9ec823/redis-module-0.26.0/src/lib.rs:77:33 1: intervalsets::info_func at /home/code/.cargo/registry/src/github.com-1ecc6299db9ec823/redis-module-0.26.0/src/macros.rs:120:13 2: modulesCollectInfo at src/module.c:8965:9 3: logModulesInfo at src/debug.c:1711:22 4: printCrashReport at src/debug.c:1967:5 sigsegvHandler at src/debug.c:1949:5 5: 6: pthread_kill 7: raise 8: abort 9: std::sys::unix::abort_internal at /build/rustc-jfpqmS/rustc-1.57.0+dfsg1+llvm/library/std/src/sys/unix/mod.rs:259:14 10: rust_panic at /build/rustc-jfpqmS/rustc-1.57.0+dfsg1+llvm/library/std/src/panicking.rs:674:5 11: std::panicking::rust_panic_with_hook at /build/rustc-jfpqmS/rustc-1.57.0+dfsg1+llvm/library/std/src/panicking.rs:642:5 12: std::panicking::begin_panic_handler::{{closure}} at /build/rustc-jfpqmS/rustc-1.57.0+dfsg1+llvm/library/std/src/panicking.rs:521:13 13: std::sys_common::backtrace::rust_end_short_backtrace at /build/rustc-jfpqmS/rustc-1.57.0+dfsg1+llvm/library/std/src/sys_common/backtrace.rs:139:18 14: rust_begin_unwind at /build/rustc-jfpqmS/rustc-1.57.0+dfsg1+llvm/library/std/src/panicking.rs:517:5 15: core::panicking::panic_fmt at /build/rustc-jfpqmS/rustc-1.57.0+dfsg1+llvm/library/core/src/panicking.rs:100:14 16: core::result::unwrap_failed at /build/rustc-jfpqmS/rustc-1.57.0+dfsg1+llvm/library/core/src/result.rs:1616:5 17: core::result::Result<T,E>::unwrap at /build/rustc-jfpqmS/rustc-1.57.0+dfsg1+llvm/library/core/src/result.rs:1298:23 18: ::from_str at /home/code/code/redis-interval-sets/src/structs.rs:79:72 19: intervalsets::get_load_data_as_str at /home/code/code/redis-interval-sets/src/lib.rs:52:12 20: intervalsets::rdb_load at /home/code/code/redis-interval-sets/src/lib.rs:46:17 21: rdbLoadObject at src/rdb.c:2597:21 22: rdbLoadRioWithLoadingCtx at src/rdb.c:3050:15 23: rdbLoadRio at src/rdb.c:2822:18 rdbLoad at src/rdb.c:3199:14 24: loadDataFromDisk at src/server.c:6396:13 25: main at src/server.c:6859:9 26: 27: __libc_start_main 28: _start `

so adding some console logging in the import loop, and it shows this when printing i i = A','min_score':37227008,'max_score':37227263},{'member':'us*@','min_score':37227264,'max_score':37227519} found: min_member => 37227008 found: max_member => 37227263} thread '<unnamed>' panicked at 'called Result::unwrap() on an Err value: ParseIntError { kind: InvalidDigit }', src/structs.rs:85:46

Notice 37227263} having a curly bracket?

i was wondering where this was coming from. I wonder if its because I'm binary packing the member (using php pack function)

here, the member is A - 6 characters ( one invisible) . This is two letter country code concated with a binary packed 32 bit integer representing an ASN, here 213034 (x0003402A). This is done to reduce memory usage when the ASN is a 16 bit integer as the member string length is 2 bytes instead of a possible 5. If I change the member from a binary representation to an ASCII one then no segfault happens. Memory usage increases a little, as expected

i shouldnt imagine a lot of people would be using non-ascii member names but you never know, and redis has always been binary safe. maybe split the member name off and then trim the min/max scores?

edit - just fixing some data as its giving me strange results on each run. I'll get a reportable command to you ASAP

stopforumspam commented 2 years ago

OK, I think I have it.

A member with a binary encoded member that has a } in it, one that is then followed by another member with a } in it, will generate invalid JSON (in the loop for i in s.chars() ) that is then parsed, causing the panic

processing i = A','min_score':37227008,'max_score':37227263},{'member':'us}

if you can, you might want to look at using msgpack libs for handling all that string processing, as having a member name with a } will break your character parsing/building. having a member name with a ' character in it will have that stripped, resulting in data integrity failure.

attached in a dump that will segfault on load dump.zip

TL/DR - you cant have a { or } in member names

danitseitlin commented 2 years ago

i've just started testing, so far so good :)

I didn't merge the fix yet, tests are throwing some errors getting help by the original redismodule-rs lib contributors :) Will message you here when finally merged!

stopforumspam commented 2 years ago

i cherry picked the commits in your testing branch. it saves/loads OK with ascii member names, but I havent done any serious load testing on it yet

danitseitlin commented 2 years ago

@stopforumspam Cool, thanks! I will keep you updated on when it's merged :)