search

opnsense / ports

OPNsense ports on top of FreeBSD
https://opnsense.org/
Other
157 stars 112 forks source link

os-mdns-repeater: hammers syslog if one of the repeated networks has no device connected #142

Closed triantos closed 2 years ago

triantos commented 2 years ago

Important notices Before you add a new report, we ask you kindly to acknowledge the following:

Describe the bug If os-mdns-repeater is enabled to repeat across a few networks and one of them doesn't have equipment connected to the interface, the plugin will bombard system.log with errors that the network is down (I was seeing roughly one message per second). The net result of this is the disk will fill up, and the CPU will be hammered (waiting on I/O) by the syslog-ng process trying to write all the messasges to disk.

To Reproduce Steps to reproduce the behavior:

  1. Install plugin os-mdns-repeater
  2. Add one interface with nothing plugged into it (e.g. OPT1)
  3. In mdns-repeater setup, have it repeat bonjour packets to the OPT1 interface plus another (e.g. LAN)
  4. SSH into the box, and watch the output to /var/log/system/latest.log

What I see is constant messages of the form:

<27>1 2022-02-25T23:59:51-08:00 opnsense.lan mdns-repeater 43304 - [meta sequenceId="55385"] send(): Network is down
<27>1 2022-02-25T23:59:51-08:00 opnsense.lan mdns-repeater 43304 - [meta sequenceId="55386"] send(): Network is down
<27>1 2022-02-25T23:59:51-08:00 opnsense.lan mdns-repeater 43304 - [meta sequenceId="55387"] send(): Network is down

Expected behavior Maybe nothing, or maybe complain once that an interface has nothing connected.

Relevant log files See the 4 lines pasted from above

Additional context Add any other context about the problem here.

Environment OPNsense 22.1.1_3-amd64 FreeBSD 13.0-STABLE OpenSSL 1.1.1m 14 Dec 2021

Running on a Supermicro board with 4 NICs and CPU Intel(R) Pentium(R) CPU N3700 @ 1.60GHz (4 cores, 4 threads)

triantos commented 2 years ago

In my case the box became nearly unusable as a result (due to CPU + i/o overload). I shelled into the box and manually killed mdns-repeater to give myself enough cycles to troubleshoot and fix this problem.

fichtner commented 2 years ago

@triantos thanks for your analysis. It looks like this should be prevented in the software itself, not the plugin. mdns-repeater, however, is not really maintained so the question is where to report this / carry a fix. Do you agree we should be avoiding the excessive logging in the daemon itself? If yes I would move the ticket to ports.git.

triantos commented 2 years ago

Yes my opinion is this is a bug in the actual daemon. If it's really unmaintained then perhaps Opnsense can put a warning in the plugin that "disconnected adapters may cause significant logging" or something like that.

thank you!

fichtner commented 2 years ago

@triantos I moved the "logging" to foreground mode... it's a useful error when debating it but logging it for every packet seems like the wrong strategy here, see 71f7612

triantos commented 2 years ago

This seems like a better approach. thanks very much.