search

etaty / rediscala

Non-blocking, Reactive Redis driver for Scala (with Sentinel support)
Apache License 2.0
788 stars 142 forks source link

possible memory leak? #147

Open redlion99 opened 8 years ago

redlion99 commented 8 years ago

I found many redis Operation objects remains in memory, while I stop sending anything through the redis connection for a long time.

Here is the memory usage:

2016-07-11 7 41 20

Memory usage histogram from jmap:

root@collector-test-79:/# /opt/java8/bin/jmap -histo 8 | grep redis 50: 96331 2311944 redis.Operation 60: 40322 1612880 redis.api.hashes.Hset 78: 20177 807080 redis.api.keys.Expire 79: 20122 804880 redis.api.lists.Lpush 84: 15707 628280 redis.api.hashes.Hget 133: 9789 234936 redis.protocol.FullyDecoded 281: 2895 46320 redis.Operation$$anonfun$decodeRedisReplyThenComplete$1 282: 2894 46304 redis.protocol.Integer 283: 2893 46288 redis.protocol.RedisProtocolReply$$anonfun$decodeInteger$1 498: 356 8544 [Lredis.Operation; 500: 356 8544 redis.protocol.RedisProtocolRequest$$anonfun$multiBulk$2 571: 356 5696 redis.actors.QueuePromises 947: 89 1424 redis.protocol.Bulk 948: 89 1424 redis.protocol.RedisProtocolReply$$anonfun$decodeBulk$1 988: 88 1408 redis.api.lists.Lpush$$anonfun$1 1013: 80 1280 redis.RedisCommandBulkOptionByteString$$anonfun$decodeReply$1 1100: 11 880 redis.actors.RedisClientActor 1104: 36 864 redis.RoundRobinPoolRequest$$anonfun$send$2 1218: 36 576 redis.RoundRobinPoolRequest$$anonfun$send$1 1374: 11 352 redis.actors.RedisReplyDecoder 1421: 10 320 redis.RedisServer 1550: 10 240 redis.RedisClientPoolLike$$anonfun$getConnectOperations$1 1551: 10 240 redis.RedisClientPoolLike$$anonfun$onConnectStatus$1 1552: 10 240 redis.RedisConnection 1674: 11 176 redis.actors.RedisClientActor$$anonfun$1 1675: 11 176 redis.actors.RedisClientActor$$anonfun$writing$1 1676: 11 176 redis.actors.RedisReplyDecoder$$anonfun$receive$1 1677: 11 176 redis.actors.RedisWorkerIO$$anonfun$reading$1 2348: 1 56 redis.ByteStringSerializer$ 2349: 1 56 redis.RedisClient 2350: 1 56 redis.protocol.RedisProtocolReply$ 2520: 1 48 redis.RedisClientPool 2521: 2 48 redis.protocol.PartiallyDecoded 2650: 1 40 redis.api.sets.Sadd 2651: 1 40 redis.api.strings.Set 2959: 1 32 redis.ByteStringDeserializer$ 2960: 1 32 redis.api.keys.Keys 3246: 1 24 redis.Redis$ 3247: 1 24 redis.protocol.DecodeResult$$anonfun$foreach$1 3248: 1 24 redis.protocol.RedisProtocolRequest$ 4365: 1 16 redis.ByteStringDeserializerDefault$RedisDouble$ 4366: 1 16 redis.ByteStringDeserializerDefault$String$ 4367: 1 16 redis.ByteStringSerializerLowPriority$IntConverter$ 4368: 1 16 redis.ByteStringSerializerLowPriority$LongConverter$ 4369: 1 16 redis.ByteStringSerializerLowPriority$String$ 4370: 1 16 redis.MultiBulkConverter$ 4371: 1 16 redis.MultiBulkConverter$$anonfun$toSeqString$1 4372: 1 16 redis.MultiBulkConverter$$anonfun$toSeqString$1$$anonfun$apply$1 4373: 1 16 redis.MultiBulkConverter$$anonfun$toSeqString$2 4374: 1 16 redis.RedisClient$ 4375: 1 16 redis.RedisClientActorLike$$anonfun$getConnectOperations$1 4376: 1 16 redis.RedisClientActorLike$$anonfun$onConnectStatus$1 4377: 1 16 redis.RedisClientPool$ 4378: 1 16 redis.RedisDispatcher 4379: 1 16 redis.RedisServer$ 4380: 1 16 redis.actors.KillOldRepliesDecoder$ 4381: 1 16 redis.actors.WriteAck$ 4382: 1 16 redis.api.sets.Sadd$$anonfun$1 4383: 1 16 redis.protocol.DecodeResult$ 4384: 1 16 redis.protocol.Integer$ 4385: 1 16 redis.protocol.MultiBulk 4386: 1 16 redis.protocol.ParseNumber$ 4387: 1 16 redis.protocol.RedisProtocolReply$$anonfun$1 4388: 1 16 redis.protocol.RedisProtocolReply$$anonfun$1$$anonfun$applyOrElse$1 4389: 1 16 redis.protocol.RedisProtocolReply$$anonfun$2 4390: 1 16 redis.protocol.RedisProtocolReply$$anonfun$3 4391: 1 16 redis.protocol.RedisProtocolReply$$anonfun$4 4392: 1 16 redis.protocol.RedisProtocolReply$$anonfun$5 4393: 1 16 redis.protocol.RedisProtocolReply$$anonfun$6 4394: 1 16 redis.protocol.RedisProtocolReply$$anonfun$decodeMultiBulk$1 4395: 1 16 redis.protocol.RedisProtocolReply$$anonfun$redis$protocol$RedisProtocolReply$$decodeBulkBody$1$1 4396: 1 16 redis.protocol.Status 4397: 1 16 redis.protocol.Status$

farico commented 7 years ago

I had a similar issue with pfadd, turns out I was sending too much commands to redis server and since they started lagging to complete I observed so much instances of Redis.pfadd. Resolved by throttling commands sent to Redis server.