search

spender-sandbox / cuckoo-modified

Modified edition of cuckoo
395 stars 178 forks source link

help re elasticsearch 'searchonly' JVM heap and big docs issues? #416

Open mallorybobalice opened 7 years ago

mallorybobalice commented 7 years ago

is anyone familiar enough to suggest what to do re java heap config and re large docs or long http req lines . I'm trying to understand if JVM heap messages are more 'change a config setting' (I'm a bit lost as to where elastic stores it) or 'you configured too much for mongo + elastic in search only + too many works.

again eager to hear if people can comment on running Elastic 5 only as main reporting DB 

[2017-02-14 16:33:22,425][WARN ][netty.channel.socket.nio.AbstractNioSelector] Unexpected exception in the selector loop.
java.lang.OutOfMemoryError: Java heap space
[2017-02-14 16:33:18,476][WARN ][netty.channel.socket.nio.AbstractNioSelector] Unexpected exception in the selector loop.
java.lang.OutOfMemoryError: Java heap space
[2017-02-14 16:33:14,276][WARN ][netty.channel.socket.nio.AbstractNioSelector] Unexpected exception in the selector loop.
java.lang.OutOfMemoryError: Java heap space
[2017-02-14 16:31:53,331][WARN ][http.netty               ] [Bucky III] Caught exception while handling client http traffic, closing connection [id: 0x479530fc, /xxx.xxx.xxx.xxx:47048 :> /127.0.0.1:9200]
java.lang.OutOfMemoryError: Java heap space
[2017-02-14 16:35:53,469][WARN ][netty.channel.socket.nio.AbstractNioSelector] Unexpected exception in the selector loop.
java.lang.OutOfMemoryError: Java heap space
[2017-02-14 16:34:37,940][WARN ][netty.channel.socket.nio.AbstractNioSelector] Unexpected exception in the selector loop.
java.lang.OutOfMemoryError: Java heap space
[2017-02-14 16:34:26,468][WARN ][netty.channel.socket.nio.AbstractNioSelector] Unexpected exception in the selector loop.
java.lang.OutOfMemoryError: Java heap space
[2017-02-14 16:34:20,432][WARN ][netty.channel.socket.nio.AbstractNioSelector] Unexpected exception in the selector loop.
java.lang.OutOfMemoryError: Java heap space
[2017-02-14 16:38:17,718][WARN ][netty.channel.DefaultChannelPipeline] An exception was thrown by a user handler while handling an exception event ([id: 0xbe199dc3, /xxx.xxx.xxx.xxx:48730 => /127.0.0.1:9200] EXCEPTION: java.lang.OutOfMemoryError: Java heap space)

[2017-02-14 16:38:43,279][WARN ][http.netty               ] [Bucky III] Caught exception while handling client http traffic, closing connection [id: 0x9a20c4c4, /xxx.xxx.xxx.xxx:48772 :> /127.0.0.1:9200]
org.jboss.netty.handler.codec.frame.TooLongFrameException: An HTTP line is larger than 4096 bytes.
        at org.jboss.netty.handler.codec.http.HttpMessageDecoder.readLine(HttpMessageDecoder.java:670)
        at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:184)
        at org.jboss.netty.handler.codec.http.HttpMessageDecoder.decode(HttpMessageDecoder.java:102)
        at org.jboss.netty.handler.codec.replay.ReplayingDecoder.callDecode(ReplayingDecoder.java:500)

[2017-02-14 16:38:43,531][WARN ][indices.memory           ] [Bucky III] failed to set shard [cuckoo-2017-02-14][1] index buffer to [32.9mb]
org.apache.lucene.store.AlreadyClosedException: translog [30] is already closed
        at org.elasticsearch.index.translog.TranslogWriter.ensureOpen(TranslogWriter.java:329)
        at org.elasticsearch.index.translog.BufferingTranslogWriter.updateBufferSize(BufferingTranslogWriter.java:143)

[2017-02-14 16:38:46,007][WARN ][indices.memory           ] [Bucky III] failed to set shard [cuckoo-2017-02-14][1] index buffer to [24.7mb]
org.apache.lucene.store.AlreadyClosedException: translog [30] is already closed
        at org.elasticsearch.index.translog.TranslogWriter.ensureOpen(TranslogWriter.java:329)
        at org.elasticsearch.index.translog.BufferingTranslogWriter.updateBufferSize(BufferingTranslogWriter.java:143)
        at org.elasticsearch.index.translog.Translog.updateBuffer(Translog.java:406)
apt list --installed|egrep -i elastic
WARNING: apt does not have a stable CLI interface. Use with caution in scripts.
elasticsearch/unknown,now 2.4.0 all [installed,upgradable to: 2.4.4]

/usr/bin/java -Xms256m -Xmx1g -Djava.awt.headless=true -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -XX:+HeapDumpOnOutOfMemoryError -XX:+DisableExplicitGC -Dfile.encoding=UTF-8 -Djna.nosys=true -Des.path.home=/usr/share/elasticsearch -cp /usr/share/elasticsearch/lib/elasticsearch-2.4.0.jar:/usr/share/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch start -Des.pidfile=/var/run/elasticsearch/elasticsearch.pid -Des.default.path.home=/usr/share/elasticsearch -Des.default.path.logs=/var/log/elasticsearch -Des.default.path.data=/var/lib/elasticsearch -Des.default.path.conf=/etc/elasticsearch

test box is running 5, just never getting the time to upgrade [everything is localhost access only]