CPU usage on Ubuntu 14.04 (knockd 0.5)

[gazilla]

I have been using knockd on Ubuntu servers (KVM-QEMU) for years and have never experienced any excess CPU usage (usually 0%). As at Ubuntu 14.04 (trusty) my servers now show a consistent 2-3% of CPU for knockd. Why? Bug?

[visit1985]

Observing the same on Arch. I think this is related to libpcap, but I don't know for sure.

[MeFri]

just tested on gentoo : libpcap-1.3.0 works fine, but once you reload knockd with libpcap-1.5.3 you'll see higher cpu usage (lots of interrupts / context switches)

[jvinet]

Does this still occur with libpcap 1.6?

[visit1985]

Yes, still there on Arch.

[gazilla]

I'm still running knockd on Ubuntu 14.04 VMs (i686 & amd64), so still have libpcap-1.5.3 ...and still a problem.

[gazilla]

I just remembered I have a test VM (qemu-kvm) running Ubuntu 14.10 amd64. It runs libpcap-1.6.2 ...still a problem.

[EfficiencyVI]

Confirmed for fresh install on Ubuntu 14.04. Idk which libpcap version but it is a bit annoying as it is above most apache processes.

// Edit libpcap 1.5.3-2

[crifi]

I'm using knockd on a Debian Jessie machine with knockd-0.5-3 and libpcap-1.6.2-2. I've the same cpu issues here.

[kleest]

Can confirm this for:

1. Ubuntu 14.04 Vivid: -- libpcap = 1.6.2-2 -- knockd = 0.5-3ubuntu1
2. Debian 8 (Jessie) -- libpcap = 1.6.2-2 -- knockd = 0.5-3

[SafwatHalaby]

Can confirm for Debian 8 (jessie): Libpcap 1.6.2-2 knockd 0.5-3

[airwoflgh]

Given this looks to be a libpcap issue, should this be closed and issue raised with libpcap?

P.

[visit1985]

The CPU usage is reasonable, as it is constantly inspecting all network packages for the defined filter. I think thats the real issue here. Isn't there a better way to get notified on such an event?

[infernix]

I wouldn't say it's reasonable, there is a big jump in CPU time spent with newer libpcap versions.

I ran a perf trace against it at some point but did not capture the data. I will test against libpcap 1.3.0 and 1.7.4 and compare (currently running against 1.6.2).

[infernix]

I ran a git bisect and the commit introducing this issue is https://github.com/the-tcpdump-group/libpcap/commit/ee4085152260466ea845d9e9109a251a39ded93b which is related to https://github.com/the-tcpdump-group/libpcap/issues/335

I didn't bisect knockd but this looks to be fixed in https://github.com/jvinet/knock/commit/67748266b38b6ff3c6650e20def71800ab161974 and so the conclusion is simple:

Upgrade to 0.7 if you have high cpu usage on knockd and you are on libpcap >= 1.5.3.

Edit: There is already bug for the out of date package in BTS: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=761853 and there's a non-maintainer version being kept here: https://anonscm.debian.org/cgit/collab-maint/knock.git

[SafwatHalaby]

The CPU usage is reasonable, as it is constantly inspecting all network packages for the defined filter. I think thats the real issue here. Isn't there a better way to get notified on such an event?

It is not reasonable because the usage is there even when there is no network activity. And it wasn't always like that.

[infernix]

@LogicParrot see my last comment; TL;DR upgrade to 0.7 where this issue is fixed.

[chris001]

@infernix Also experiencing high system load from knock on ubuntu 14.04. I download thie zip of 0.7 from this github, run autoconf and automake and got a page full of errors. Must be missing something. Any suggestion how to cleanly upgrade this to 0.7 ?

[infernix]

@chris001 git clone git://anonscm.debian.org/collab-maint/knock.git && cd knock && dpkg-buildpackage -b

install any missing dependencies if it complains.

[chris001]

@infernix thanks because downloading and unzipping the source from this github repo and running aclocal && autoconf && automake --add-missing gives error: required file 'config.h.in' not found

[airwoflgh]

I've just downloaded the master branch from Github and successfully autoconf'd and compiled it on OSX and CentOS 6.7. Do you have libpcap headers installed and using the master branch and not one of the dev trees?

If you are using my own personal one, it isn't ready for compilation.

P.

On 24 May 2016 10:51:50 GMT-04:00, Gerben Meijer notifications@github.com wrote:

@chris001 git clone git://anonscm.debian.org/collab-maint/knock.git && cd knock && dpkg-buildpackage -b

install any missing dependencies if it complains.

---

You are receiving this because you commented. Reply to this email directly or view it on GitHub: https://github.com/jvinet/knock/issues/9#issuecomment-221296428

[chris001]

@airwoflgh I downloaded the zip from master branch here. No libpcap headers installed so that's probably why the file not found error. But that's OK though, because....

I ran the commands suggested by @infernix with a few additional ones... sudo apt-get install dh-reconfigure git clone git://anonscm.debian.org/collab-maint/knock.git cd knock dpkg-buildpackage -b sudo service knockd stop sudo make install sudo service knockd start

..and the excessive system load, caused by knockd 0.5, quickly drops from 0.45 to 0.01with this new version knockd 0.7 ! The system is back to working fast like it used to. No more lag. Thanks again!

[airwoflgh]

Great! Glad it is resolved.

P.

On 24 May 2016 11:18:46 GMT-04:00, Chris Coleman notifications@github.com wrote:

@airwoflgh I downloaded the zip from master branch. No libpcap headers installed so that's probably why the file not found error. But that's OK thought because....

I ran the commands suggested by @infernix with a few additional ones... sudo apt-get install dh-reconfigure git clone git://anonscm.debian.org/collab-maint/knock.git cd knock dpkg-buildpackage -b sudo service knockd stop sudo make install sudo service knockd start

..and the excessive system load, caused by knockd 0.5, quickly drops from 0.45 to 0.01with knockd 0.7 ! The system is back to working fast like it used to. No more lag. Thanks again!

---

You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/jvinet/knock/issues/9#issuecomment-221305198

Spam detection software, running on the system "pootle.flumps.org", has identified this incoming email as possible spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see postmaster@flumps.org for details.

Content preview: Great! Glad it is resolved. P. On 24 May 2016 11:18:46 GMT-04:00, Chris Coleman notifications@github.com wrote: >@airwoflgh I downloaded the zip from master branch. No libpcap >headers installed so that's probably why the file not found error. But >that's OK thought because.... > >I ran the commands suggested by @infernix with a few additional ones... >`sudo apt-get install dh-reconfigure >git clone git://anonscm.debian.org/collab-maint/knock.git

cd knock >dpkg-buildpackage -b >sudo service knockd stop >sudo make install sudo service knockd start` > >..and the excessive system load, caused by knockd 0.5, quickly drops >from 0.45 to 0.01with knockd 0.7 ! >The system is back to working fast like it used to. No more lag. >Thanks again! > >--- You are receiving this because you were mentioned. >Reply to this email directly or view it on GitHub: >https://github.com/jvinet/knock/issues/9#issuecomment-221305198 [...]

Content analysis details: (3.6 points, 3.0 required)

pts rule name description

---

1.0 URIBL_BLOCKED ADMINISTRATOR NOTICE: The query to URIBL was blocked. See http://wiki.apache.org/spamassassin/DnsBlocklists#dnsbl-block for more information. [URIs: github.com] 0.7 HTML_IMAGE_ONLY_20 BODY: HTML: images with 1600-2000 bytes of words 0.0 HTML_MESSAGE BODY: HTML included in message 1.9 RDNS_NONE Delivered to internal network by a host with no rDNS 0.0 T_REMOTE_IMAGE Message contains an external image 0.0 HELO_MISC_IP Looking for more Dynamic IP Relays

[infernix]

@chris001 FWIW what dpkg-buildpackage does is generate a .deb package which you can (and should) install instead of doing make install

[chris001]

@infernix thanks for this tip, a .deb package would be awesome but I don't see the .deb anywhere in the knock folder tree?

Anyway, I've contacted the debian maintainer of knockd, Leo Costelo, suggesting he add 0.7 to wheezy 7, jessie 8, stretch 9, and sid unstable, all of which are currently on 0.5-3 which is prone to high CPU when libpcap is >= 1.5.3, which it is.

[airwoflgh]

Appreciate you doing that Chris. Hadn't dug into why people were still using an old version.

Will check some other major distributions....

On 24 May 2016 11:46:30 GMT-04:00, Chris Coleman notifications@github.com wrote:

@infernix thanks for this tip, a .deb package would be awesome but I don't see the .deb anywhere in the knock folder tree?

Anyway, I've contacted the debian maintainer of knockd, Leo Costelo, suggesting he add 0.7 to wheezy 7, jessie 8, stretch 9, and sid unstable, all of which are currently on 0.5-3 which is prone to high CPU when libpcap is >= 1.5.3, which it is.

---

You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/jvinet/knock/issues/9#issuecomment-221314430

[infernix]

@chris001 dpkg-buildpackge, if ran successfully, outputs the generated deb path on stdout (see ../)

[chris001]

@infernix you are correct the debs are one directory higher. So to fix my sudo make install knockd which caused the problem of the system package manager apt thinking it still had 0.5 installed when in fact 0.7 is installed, I had to do sudo dpkg -u knockd sudo dpkg -u knock to remove knockd and knock so the system thinks 0.5 is removed, then, dpkg -i knock_0.7-1_i386.deb dpkg -i knockd_0.7-1_i386.deb to install the new versions.

FYI knockd only will install if knock is already installed and the same or greater version.

[chris001]

Wow that's weird. It appears that the code on anonscm.debian.org has been removed offline. The git clone command no longer works. Browse there and you'll see yourself. Possibly Leo Costelo received my email and removed knock 0.7 from there.

[chris001]

@airwoflgh Is it possible Leo Costelo or one of the maintainers you contacted, has removed the contents of anonscm.debian.org because it's still showing:

Not Found The requested URL /collab-maint/knock.git was not found on this server. Apache/2.2.22 (Debian) Server at anonscm.debian.org Port 80

[airwoflgh]

I haven't reached out to anyone at Debian as I took from your email that you had / were going to.

P.

On 28 May 2016 15:41:55 GMT-04:00, Chris Coleman notifications@github.com wrote:

@airwoflgh Is it possible Leo Costelo or one of the maintainers you contacted, has removed the contents of anonscm.debian.org because it's still showing:

`Not Found

The requested URL /collab-maint/knock.git was not found on this server. Apache/2.2.22 (Debian) Server at anonscm.debian.org Port 80`

---

You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/jvinet/knock/issues/9#issuecomment-222326181

[chris001]

ok. Yes I'd emailed Costelo at debian, the debian maintainer, told him 0.7 fixes that high cpu bug and should be updated on debian 7, 8, 9, and sid unstable, and within an hour, the knock git repo code was removed from anonscm.debian.org so probably he removed it! The 0.7 update to debian knock ought to fix both debian and ubuntu distros since ubuntu pulls more or less every update debian publishes.

[ibaldonl]

This issue should be closed, it's fixed in Debian and not a current knockd issue, thanks.