search

antirez / neural-redis

Neural networks module for Redis
BSD 3-Clause "New" or "Revised" License
2.22k stars 108 forks source link

Redis bug report #6

Open jmgirven opened 7 years ago

jmgirven commented 7 years ago

I created a docker with the redis unstable branch and neural redis installed in it. I ran it on an AWS t2.micro instance, and tried to create a neural network just to play with. The docker crashed with the below bug report. The machine is just too small to work with it?

=== REDIS BUG REPORT START: Cut & paste starting from here === 1:M 14 Oct 15:05:57.557 # Redis 999.999.999 crashed by signal: 4 1:M 14 Oct 15:05:57.557 # Crashed running the instuction at: 0x7fd4aaff730c 1:M 14 Oct 15:05:57.557 # Failed assertion: (:0)

------ STACK TRACE ------ EIP: /usr/src/neural-redis/neuralredis.so(AnnSetDeltas+0x1c)[0x7fd4aaff730c]

Backtrace: redis-server :6379(logStackTrace+0x29)[0x463da9] redis-server :6379(sigsegvHandler+0xa6)[0x464446] /lib/x86_64-linux-gnu/libpthread.so.0(+0xf8d0)[0x7fd4ad29a8d0] /usr/src/neural-redis/neuralredis.so(AnnSetDeltas+0x1c)[0x7fd4aaff730c]

------ INFO OUTPUT ------

Server

redis_version:999.999.999 redis_git_sha1:f6332120 redis_git_dirty:1 redis_build_id:897972011416e494 redis_mode:standalone os:Linux 3.13.0-92-generic x86_64 arch_bits:64 multiplexing_api:epoll gcc_version:4.9.2 process_id:1 run_id:71b1bc746de2285bdf48e53bd19a6c1ccf5fd7e2 tcp_port:6379 uptime_in_seconds:602 uptime_in_days:0 hz:10 lru_clock:62549 executable:/data/redis-server config_file:

Clients

connected_clients:6 client_longest_output_list:0 client_biggest_input_buf:0 blocked_clients:0

Memory

used_memory:934192 used_memory_human:912.30K used_memory_rss:4571136 used_memory_rss_human:4.36M used_memory_peak:994496 used_memory_peak_human:971.19K used_memory_peak_perc:93.94% used_memory_overhead:798424 used_memory_startup:765592 used_memory_dataset:135768 used_memory_dataset_perc:80.53% total_system_memory:1040695296 total_system_memory_human:992.48M used_memory_lua:37888 used_memory_lua_human:37.00K maxmemory:0 maxmemory_human:0B maxmemory_policy:noeviction mem_fragmentation_ratio:4.89 mem_allocator:jemalloc-4.0.3 lazyfree_pending_objects:0

Persistence

loading:0 rdb_changes_since_last_save:204 rdb_bgsave_in_progress:0 rdb_last_save_time:1476456955 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:-1 rdb_current_bgsave_time_sec:-1 rdb_last_cow_size:0 aof_enabled:0 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_last_write_status:ok aof_last_cow_size:0

Stats

total_connections_received:9 total_commands_processed:811 instantaneous_ops_per_sec:7 total_net_input_bytes:31366 total_net_output_bytes:1999708 instantaneous_input_kbps:0.25 instantaneous_output_kbps:0.06 rejected_connections:0 sync_full:0 sync_partial_ok:0 sync_partial_err:0 expired_keys:0 evicted_keys:0 keyspace_hits:3 keyspace_misses:195 pubsub_channels:0 pubsub_patterns:1 latest_fork_usec:0 migrate_cached_sockets:0

Replication

role:master connected_slaves:0 master_repl_offset:0 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0

CPU

used_cpu_sys:0.25 used_cpu_user:0.31 used_cpu_sys_children:0.00 used_cpu_user_children:0.00

Commandstats

cmdstat_del:calls=1,usec=581,usec_per_call=581.00 cmdstat_zadd:calls=3,usec=42,usec_per_call=14.00 cmdstat_multi:calls=198,usec=276,usec_per_call=1.39 cmdstat_exec:calls=594,usec=4179,usec_per_call=7.04 cmdstat_psubscribe:calls=2,usec=7,usec_per_call=3.50 cmdstat_publish:calls=11,usec=26,usec_per_call=2.36 cmdstat_command:calls=2,usec=633,usec_per_call=316.50

Cluster

cluster_enabled:0

Keyspace

hash_init_value: 1476991743

------ CLIENT LIST OUTPUT ------ id=4 addr=127.0.0.1:45054 fd=7 name= age=258 idle=97 flags=N db=0 sub=0 psub=1 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=psubscribe id=6 addr=172.17.0.3:48818 fd=6 name= age=99 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=exec id=7 addr=172.17.0.3:48820 fd=8 name= age=99 idle=97 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=publish id=8 addr=172.17.0.3:48821 fd=9 name= age=99 idle=1 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=exec id=9 addr=172.17.0.3:48822 fd=10 name= age=99 idle=99 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=0 obl=0 oll=0 omem=0 events=r cmd=publish id=10 addr=172.17.0.3:48827 fd=11 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=nr.create

------ CURRENT CLIENT INFO ------ id=10 addr=172.17.0.3:48827 fd=11 name= age=0 idle=0 flags=N db=0 sub=0 psub=0 multi=-1 qbuf=0 qbuf-free=32768 obl=0 oll=0 omem=0 events=r cmd=nr.create argv[0]: 'NR.CREATE' argv[1]: 'filters_nn' argv[2]: 'CLASSIFIER' argv[3]: '24' argv[4]: '20' argv[5]: '->' argv[6]: '14' argv[7]: 'DATASET' argv[8]: '1000' argv[9]: 'TEST' argv[10]: '500' argv[11]: 'NORMALIZE'

------ REGISTERS ------ 1:M 14 Oct 15:05:57.576 # RAX:0000000000000040 RBX:0000000000000003 RCX:00000000017ea530 RDX:0000000000000003 RDI:00000000017e8500 RSI:00000000017ea4b0 RBP:00000000017e8500 RSP:00007ffde85eabc0 R8 :00007fd4ad2850e8 R9 :00007fd4ad285120 R10:00007ffde85eabe0 R11:00000000017ecf00 R12:0000000000000003 R13:00007ffde85eacc0 R14:00000000000001f4 R15:00007fd4ab1fbbd0 RIP:00007fd4aaff730c EFL:0000000000010202 CSGSFS:0000000000000033 1:M 14 Oct 15:05:57.577 # (00007ffde85eabcf) -> 00007fd4ab1fbaf8 1:M 14 Oct 15:05:57.577 # (00007ffde85eabce) -> 000000000000000c 1:M 14 Oct 15:05:57.577 # (00007ffde85eabcd) -> 00007fd4ad005a10 1:M 14 Oct 15:05:57.577 # (00007ffde85eabcc) -> 00007fd4aca23080 1:M 14 Oct 15:05:57.577 # (00007ffde85eabcb) -> 00007ffde85eacc0 1:M 14 Oct 15:05:57.577 # (00007ffde85eabca) -> 000003e8e85eada0 1:M 14 Oct 15:05:57.577 # (00007ffde85eabc9) -> 00007fd4aaff17cf 1:M 14 Oct 15:05:57.577 # (00007ffde85eabc8) -> 0000000000000003 1:M 14 Oct 15:05:57.577 # (00007ffde85eabc7) -> 00007fd4acb30000 1:M 14 Oct 15:05:57.577 # (00007ffde85eabc6) -> 00007ffde85eac50 1:M 14 Oct 15:05:57.577 # (00007ffde85eabc5) -> 000000000000000c 1:M 14 Oct 15:05:57.577 # (00007ffde85eabc4) -> 0000000000000088 1:M 14 Oct 15:05:57.577 # (00007ffde85eabc3) -> 00007fd4aaff7738 1:M 14 Oct 15:05:57.577 # (00007ffde85eabc2) -> 00007fd4ab1fbbd0 1:M 14 Oct 15:05:57.577 # (00007ffde85eabc1) -> 00000000000001f4 1:M 14 Oct 15:05:57.577 # (00007ffde85eabc0) -> 00007ffde85eacc0

------ FAST MEMORY TEST ------ 1:M 14 Oct 15:05:57.577 # Bio thread for job type #0 terminated 1:M 14 Oct 15:05:57.578 # Bio thread for job type #1 terminated 1:M 14 Oct 15:05:57.578 # Bio thread for job type #2 terminated * Preparing to test memory region 744000 (94208 bytes) * Preparing to test memory region 17de000 (135168 bytes) * Preparing to test memory region 7fd4ab1fc000 (4096 bytes) * Preparing to test memory region 7fd4ab1fe000 (8388608 bytes) * Preparing to test memory region 7fd4ab9ff000 (8388608 bytes) * Preparing to test memory region 7fd4ac200000 (12582912 bytes) * Preparing to test memory region 7fd4ad287000 (16384 bytes) * Preparing to test memory region 7fd4ad4a4000 (16384 bytes) * Preparing to test memory region 7fd4adbc4000 (16384 bytes) * Preparing to test memory region 7fd4adbc9000 (16384 bytes) *\ Preparing to test memory region 7fd4adbcf000 (4096 bytes) .O.O.O.O.O.O.O.O.O.O.O Fast memory test PASSED, however your memory can still be broken. Please run a memory test for several hours if possible.

------ DUMPING CODE AROUND EIP ------ Symbol: AnnSetDeltas (base: 0x7fd4aaff72f0) Module: /usr/src/neural-redis/neuralredis.so (base 0x7fd4aafec000) $ xxd -r -p /tmp/dump.hex /tmp/dump.bin

$ objdump --adjust-vma=0x7fd4aaff72f0 -D -b binary -m i386:x86-64 /tmp/dump.bin

1:M 14 Oct 15:05:57.690 # dump of function (hexdump of 156 bytes): 8b470483f8010f8eb40100004c8d5424084883e4e083e80248c1e006c4e27d18c841ff72f8554889e54154415253488b5720488d7a784c8d8c02b800000066908b57880faf57c885d20f8e57010000488b0f4889c883e01f48c1e80248f7d883e00739d00f47c283fa0a0f8f5801000089d083f801c5fa11010f846101000083f802c5fa1141040f84d301000083f803c5fa1141080f84b501000083

=== REDIS BUG REPORT END. Make sure to include from START to END. ===

Dockerfile largely copied from the standard redis 3.2 dockerfile:

FROM debian:jessie

# TAKEN FROM:
# https://github.com/docker-library/redis/blob/a38166e6f3430512ba8ce2cb5ebd889ee17b9dc4/3.2/Dockerfile
# add our user and group first to make sure their IDs get assigned consistently, regardless of whatever dependencies get added
RUN groupadd -r redis && useradd -r -g redis redis

RUN apt-get update && apt-get install -y --no-install-recommends \
        ca-certificates \
        wget \
    && rm -rf /var/lib/apt/lists/*

# grab gosu for easy step-down from root
ENV GOSU_VERSION 1.7
RUN set -x \
    && wget -O /usr/local/bin/gosu "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture)" \
    && wget -O /usr/local/bin/gosu.asc "https://github.com/tianon/gosu/releases/download/$GOSU_VERSION/gosu-$(dpkg --print-architecture).asc" \
    && export GNUPGHOME="$(mktemp -d)" \
    && gpg --keyserver ha.pool.sks-keyservers.net --recv-keys B42F6819007F00F88E364FD4036A9C25BF357DD4 \
    && gpg --batch --verify /usr/local/bin/gosu.asc /usr/local/bin/gosu \
    && rm -r "$GNUPGHOME" /usr/local/bin/gosu.asc \
    && chmod +x /usr/local/bin/gosu \
    && gosu nobody true

# Common dependencies
RUN apt-get update && apt-get install -y \
        gcc \
        libc6-dev \
        make \
        git \
    && rm -rf /var/lib/apt/lists/*

# Github redis unstable install
RUN mkdir -p /usr/src \
    && git clone https://github.com/antirez/redis.git /usr/src/redis \
    \
# Disable Redis protected mode [1] as it is unnecessary in context
# of Docker. Ports are not automatically exposed when running inside
# Docker, but rather explicitely by specifying -p / -P.
# [1] https://github.com/antirez/redis/commit/edd4d555df57dc84265fdfb4ef59a4678832f6da
    && grep -q '^#define CONFIG_DEFAULT_PROTECTED_MODE 1$' /usr/src/redis/src/server.h \
    && sed -ri 's!^(#define CONFIG_DEFAULT_PROTECTED_MODE) 1$!\1 0!' /usr/src/redis/src/server.h \
    && grep -q '^#define CONFIG_DEFAULT_PROTECTED_MODE 0$' /usr/src/redis/src/server.h \
# for future reference, we modify this directly in the source instead of just supplying a default configuration flag because apparently "if you specify any argument to redis-server, [it assumes] you are going to specify everything"
# see also https://github.com/docker-library/redis/issues/4#issuecomment-50780840
# (more exactly, this makes sure the default behavior of "save on SIGTERM" stays functional by default)
    \
    && make -C /usr/src/redis \
    && make -C /usr/src/redis install \
    \
    && rm -r /usr/src/redis

# Neural redis install
RUN mkdir -p /usr/src \
    && git clone https://github.com/antirez/neural-redis.git /usr/src/neural-redis \
    && make -C /usr/src/neural-redis avx

# Remove common dependencies
RUN apt-get purge -y --auto-remove \
        gcc \
        libc6-dev \
        make \
        git

RUN mkdir /data && chown redis:redis /data
VOLUME /data
WORKDIR /data

COPY docker-entrypoint.sh /usr/local/bin/
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
ENTRYPOINT ["docker-entrypoint.sh"]

EXPOSE 6379
CMD [ "redis-server", "--loadmodule", "/usr/src/neural-redis/neuralredis.so" ]
jmgirven commented 7 years ago

It seems happier on a t2.medium (4G RAM).