Nyzo verifier stop tracking the chain "Verifier-mainLoop" java.lang.OutOfMemoryError: unable to create new native thread"

[reb0rn21]

Nyzo version 611 CE edition high speed 2 core VPS (KVM) 1GB ram DDR4 + 1GB swap ubuntu 20.04 openjdk version "1.8.0_312"

I tried to change java verifier conf from 3GB to 1GB and 768MB to limit memory use to no help, also I have more VPS with same provider and same conf and no issue

node join or some spam make nyzo java to crash, verifier still work but CPU load after crash is 100% on one core and no new block is being produced:

`^^^^^^^^^^^^^^^^^^^^^ casting vote for height 15672008 broadcasting message: BlockVote19 to 2623 added new out-of-cycle node to queue: 781f...49d6 removed node from new out-of-cycle queue due to size nodejoin_from 66.70.130.114 781f...49d6 Disgusted8uWfG storing new vote, height=15672008, hash=c4b5...f134 freezing block [Block: v=2, height=15672008, hash=c4b5...f134, id=4f93...1820] with standard mechanism after registering block [Block: v=2, height=15672008, hash=c4b5...f134, id=4f93...1820] in BlockchainMetricsManager(), cycleTransactionSum=∩15,147.169596, cycleLength=2584 BlockVoteManager: removing vote map of size 2583 cleaning up because block 15672008 was frozen sending verifier-removal vote to node at IP: 157.90.168.189 top verifier 9365...7aab has 2427 votes with a cycle length of 2584 (93.9%) added new out-of-cycle node to queue: ba29...5a52 removed node from new out-of-cycle queue due to size nodejoin_from 3.80.0.193 ba29...5a52 Siva8 maximum cycle transaction amount=∩100000.000000, balance=∩68641817.666927 maximum cycle transaction amount=∩100000.000000, balance=∩68641817.666927 transmitting block [Block: v=2, height=15672009, hash=4dd4...abe7, id=18d8...f7fd] broadcasting message: NewBlock9 to 2623

Exception in thread "Verifier-mainLoop" java.lang.OutOfMemoryError: unable to create new native thread at java.lang.Thread.start0(Native Method) at java.lang.Thread.start(Thread.java:717) at co.nyzo.verifier.Message.fetchTcp(Message.java:258) at co.nyzo.verifier.Message.fetch(Message.java:199) at co.nyzo.verifier.Message.broadcast(Message.java:148) at co.nyzo.verifier.Verifier.verifierMain(Verifier.java:419) at co.nyzo.verifier.Verifier.access$000(Verifier.java:23) at co.nyzo.verifier.Verifier$1.run(Verifier.java:260) at java.lang.Thread.run(Thread.java:748)

added new out-of-cycle node to queue: 032a...651e removed node from new out-of-cycle queue due to size nodejoin_from 3.96.79.172 032a...651e Sam_FTX 4 maximum cycle transaction amount=∩100000.000000, balance=∩68641817.666927 nodejoin_from 49.12.40.18 61e8...2367 Tik maximum cycle transaction amount=∩100000.000000, balance=∩68641817.666927 added new out-of-cycle node to queue: 0c7d...4245 removed node from new out-of-cycle queue due to size nodejoin_from 142.44.252.185 0c7d...4245 Nutka286`

[n-y-z-o]

After an out-of-memory error, the JVM will no longer be stable.

Try using a VPS with at least 2GB RAM. We use VPSes with 4GB of RAM for all of our verifiers.

[n-y-z-o]

If you want to try to run on a VPS with only 1GB of RAM, look at the maximum_concurrent_connections preference to prevent the MeshListener threads from consuming too much memory. The default value is 1000. Try adjusting it to 500.

[reb0rn21]

Ok tnx will try that too

[n-y-z-o]

Also, look at some of the other preferences at the top of MeshListener. The default values are meant to be a nice balance between being friendly to other verifiers and protective of one's own verifier.

// These values, all configurable through the preferences file, define a maximum number of connections per IP // address and a taper to reduce that maximum when the number of connections is high. The default values activate // the taper at 500 connections, reducing the number of connections per IP by 1 for every 50 additional connections // past 500. This is done with floor rounding, so the number of connections per IP would be 4 when 501 connections // are active, 3 when 551 connections are active, etc. A maximum is also defined to put an absolute cap on the // number of active connections. Whitelisted IP addresses are not subject to any of these limits, though their // connections do count toward active connections. private static final int maximumConcurrentConnectionsPerIpAbsolute = PreferencesUtil.getInt("maximum_concurrent_connections_per_ip", 5); private static final int concurrentConnectionThrottleThreshold = PreferencesUtil.getInt("concurrent_connection_throttle_threshold", 500); private static final double concurrentConnectionReductionRate = PreferencesUtil.getDouble("concurrent_connection_reduction_rate", 0.02); private static final int maximumConcurrentConnections = PreferencesUtil.getInt("maximum_concurrent_connections", 1000);

You can actually tighten these quite a bit without causing problems for anyone. Some good values might be (even lower than what I suggested previously):

maximum_concurrent_connections_per_ip=2 concurrent_connection_throttle_threshold=100 maximum_concurrent_connections=200

What this means is that you will allow up to 2 connections from each IP address until you have 100 connections active. At that point, you will allow only 1 connection from each IP. At 200 active connections, you will be refusing connections to protect your verifier. After the burst of traffic passes, you will be accepting connections again, and your verifier will be fine.