search

iotaledger / iri

IOTA Reference Implementation
Other
1.15k stars 370 forks source link

Infinite loop when neighbor DNS resolution found hostname=null #1247

Open lambtho12 opened 5 years ago

lambtho12 commented 5 years ago

Bug description

When the neighbor hostname is NULL, IRI spams the syslog (/var/log/syslog) multiple time per seconds and this fills up the whole disk (syslog file was 240Gb for a total disk space of 300). In the end, everything stops due to lack of space.

It seems like this can happen when the neighbor stops renting the VPS service (in my case at Contabo). It seems that in that case, the hostname assigned by Contabo at the domain name is NULL and that creates the continuous error reporting from IRI.

IRI version

IRI 1.5.5, installed using Nuriel's IRI-playbook

Hardware Spec

VPS from Contabo (Germany) : 6 cores, 12 GB RAM, 300 GB SSD

Steps To Reproduce

  1. Set a domain name that leads to NULL (the one that caused a problem for me was 'vmi155134.contaboserver.net')
  2. Check system logs

Expected behaviour

Avoid logging this error infinitely. Stop checking this neighbor at all if the hostname is NULL, or only check it once every few hours.

Actual behaviour

Error log was filled very quickly and everything stopped working due to lack of disk space.

Errors

Here are most of the lines that are reported in the syslog file.

Dec 19 03:55:42 vmi168191 iri[15683]: 12/19 03:55:42.066 [pool-3-thread-3] INFO  com.iota.iri.network.Node - Checking Neighbors' Ip...
Dec 19 03:55:42 vmi168191 iri[15683]: 12/19 03:55:42.066 [pool-3-thread-3] INFO  com.iota.iri.network.Node - DNS Checker: Validating DNS Address 'vmi155134.contaboserver.net' with '2a02:c207:2015:5134:0:0:0:1'
Dec 19 03:55:42 vmi168191 iri[15683]: 12/19 03:55:42.066 [pool-3-thread-3] INFO  com.iota.iri.network.Node - IP CHANGED for vmi155134.contaboserver.net! Updating...
Dec 19 03:55:42 vmi168191 iri[15683]: 12/19 03:55:42.066 [pool-3-thread-3] ERROR com.iota.iri.network.Node - Neighbor DNS Refresher Thread Exception:
Dec 19 03:55:42 vmi168191 iri[15683]: java.lang.IllegalArgumentException: hostname can't be null
lambtho12 commented 5 years ago

The errors reported in the syslog include a few other lines, but I did not copied them before removing the log file.

rajivshah3 commented 5 years ago

This is probably what caused #811 and #764