Metricbeat - Add FreeBSD support

[girgen]

This is discussed in an issue 355 at influxdata/telegraf and they suggested removing the dependency on sigar and replace it with gopsutil.

[gregoryo2014]

Following through the links, it appears that they have successfully replaced sigar with gopsutil, after handling uptime format issues (ints and floats). I'm looking forward to trying out topbeat on FreeBSD, whenever I can.

[girgen]

OK. I have a ports ready for filebeat and packetbeat. I'll try making a port of topbeat as well. If they work, I'll commit them. @gregoryo2014, do you have any opinions on whether it should be one port, say elastic-beats, or separate ports, filebeat, topbeat, packetbeat? Since there might be more beats later on, I thought that separate ports would be clever, but maybe it is not such a good idea? Suggestions?

[girgen]

Hmm, I just pulled 2bfc1c4 (master) and it still uses sigar. :-(

[tsg]

I think @gregoryo2014 was talking about telegraph? We didn't make the switch.

[gregoryo2014]

Yes, I was takling about telegraf/issues/355, per @girgen's first comment in this issue. I have a spare FreeBSD box to play on, plus some (vested) interest in this, so if someone is willing to give me some beginning pointers (URLs to read I guess) then I'll have a look at it. I'm a sysadmin though, not a dev, so if this requires deep coding cleverness it may be over my head. Perhaps I'll at least learn a thing or two - that diff list is daunting!

[gregoryo2014]

@girgen I haven't ever created a FreeBSD port, and I don't have an opinion on whether one or a few ports would be better. There might be some ports policies there which affect the decision, but if it were me I'd start with the path of least resistance, and refactor later if it proves desirable. Singing from @jordansissel's hymnal, it's all just code that can be rafactored. Easy, right? :P

[tsg]

Most of that diff is the replacing gosigar with psutil in Godeps, but I think @monicasarbu evaluated psutil when starting Topbeat and decided to use gosigar instead. We've also fixed and implemented lots of things in gosigar since then. So I'm not sure we want to just replace it or add freebsd support to it.

[gregoryo2014]

Okay, understood. I'll sit back and see what happens for a while. It's all new infrastructure for us, and a working filebeat is great so far - it seems mature enough that I won't be delving into its predecessor.

[girgen]

I have ports for filebeat and packetbeat. @gregoryo2014 would you like to test them?

[gregoryo2014]

Yes please! On 17/03/2016 11:40 pm, "Palle Girgensohn" notifications@github.com wrote:

I have ports for filebeat and packetbeat. @gregoryo2014 https://github.com/gregoryo2014 would you like to test them?

— You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub https://github.com/elastic/beats/issues/974#issuecomment-197937470

[girgen]

See if this works: @gregoryo2014 https://github.com/girgen/freebsd-ports/tree/master/sysutils/filebeat https://github.com/girgen/freebsd-ports/tree/master/sysutils/packetbeat

[gregoryo2014]

They both build and install fine. One observation: An unused cc argument.

===>  Building for packetbeat-1.1.2
cd /usr/ports/sysutils/packetbeat/work/src/github.com/elastic/beats; /usr/bin/env XDG_DATA_HOME=/usr/ports/sysutils/packetbeat/work  XDG_CONFIG_HOME=/usr/ports/sysutils/packetbeat/work  HOME=/usr/ports/sysutils/packetbeat/work NO_PIE=yes WITHOUT_DEBUG_FILES=yes WITHOUT_KERNEL_SYMBOLS=yes SHELL=/bin/sh NO_LINT=YES PREFIX=/usr/local  LOCALBASE=/usr/local  LIBDIR="/usr/lib"  CC="cc" CFLAGS="-O2 -pipe  -fstack-protector -fno-strict-aliasing"  CPP="cpp" CPPFLAGS=""  LDFLAGS=" -fstack-protector" LIBS=""  CXX="c++" CXXFLAGS="-O2 -pipe -fstack-protector -fno-strict-aliasing "  MANPREFIX="/usr/local" BSD_INSTALL_PROGRAM="install  -s -m 555"  BSD_INSTALL_LIB="install  -s -m 444"  BSD_INSTALL_SCRIPT="install  -m 555"  BSD_INSTALL_DATA="install  -m 0644"  BSD_INSTALL_MAN="install  -m 444" GOPATH="/usr/ports/sysutils/packetbeat/work:/usr/local/share/go"  CGO_CFLAGS="-I/usr/local/include"  CGO_LDFLAGS="-L/usr/local/lib"  GOBIN="" gmake -C packetbeat
gmake[1]: Entering directory '/usr/ports/sysutils/packetbeat/work/beats-1.1.2/packetbeat'
go build
# github.com/elastic/beats/vendor/github.com/tsg/gopacket/pcap
cc: warning: argument unused during compilation: '-pthread'

I'll attempt some testing from my ELK stack setup next week and let you know how it goes.

[girgen]

@gregoryo2014 great, let me know how it goes.

[gregoryo2014]

# service filebeat onestart
Starting filebeat.
# daemon: /usr/local/libexec/filebeat: No such file or directory
daemon: /usr/local/libexec/filebeat: No such file or directory
daemon: /usr/local/libexec/filebeat: No such file or directory

This repeats and fills the screen before I find and kill the PID.

[gregoryo2014]

root@klio:~ # filebeat -c /usr/local/etc/filebeat.yml -e -d "publish"

I immediately see it working equally as well as the nightlies download (data appears in Kibana), with this unsurprising difference:

root@klio:~ # filebeat -version
filebeat version 1.1.2 (amd64)
root@klio:~ # ./filebeat-freebsd-amd64 -version
filebeat version 5.0.0-nightly0424f05 (amd64)

[gregoryo2014]

This is the first time I have touched packetbeat, and so I haven't even looked at the configuration file. This is what happens when trying to start it with the default configuration file:

root@klio:~ # service packetbeat onestart
Starting packetbeat.
root@klio:~ # tail /var/log/messages
Mar 21 16:50:41 klio /usr/local/sbin/packetbeat[3704]: packetbeat.go:215: Initializing sniffer failed: Error creating sniffer: any: No such device exists (BIOCSETIF failed: Device not configured) 
Mar 21 16:50:42 klio /usr/local/sbin/packetbeat[3705]: packetbeat.go:215: Initializing sniffer failed: Error creating sniffer: any: No such device exists (BIOCSETIF failed: Device not configured) 
Mar 21 16:50:44 klio /usr/local/sbin/packetbeat[3707]: packetbeat.go:215: Initializing sniffer failed: Error creating sniffer: any: No such device exists (BIOCSETIF failed: Device not configured) 

This repeats every one or two seconds until I stop the service.

[Asara]

@gregoryo2014: In terms of https://github.com/elastic/beats/issues/974#issuecomment-199178185, You can change the init file in /usr/local/etc/rc.d/filebeat to point to /usr/local/sbin/filebeat to fix that issue. I've made a pull request

[jasperla]

@girgen @gregoryo2014 I'm working on gosigar support for OpenBSD (and thus topbeat too). You can probably base future FreeBSD support on that. Currently topbeat can report data on filesystem/memory usage and system load on OpenBSD.

[girgen]

@jasperla sounds great! When do you think it will be ready to test?

[jasperla]

@girgen whenever it's merged :) https://github.com/elastic/gosigar/pull/22 It's probably a bit rough on the edges still; i.e. might use some polishing but it's a good start.

[loganbest]

@girgen FYI on this. Works well but it has a dependency on /usr/ports/lang/go/files/bsd.go.mk which no longer exists in the latest ports tree and has to be recreated from the old tree. When will this be merged into the official ports tree, or even better pkg dist?

[SlicerDicer]

Logan, I am currently working on cleaning it up. There was a change to ports tree that moved bsd.go.mk to the referenced below.

https://github.com/freebsd/freebsd-ports/blob/master/Mk/Uses/go.mk

[gregoryo2014]

@girgen What is the status of the filebeat port for FreeBSD? I see nothing at https://bugs.freebsd.org/bugzilla/buglist.cgi?quicksearch=%22new%20port%22 mentioning elastic or beat.

@SlicerDicer @girgen It appears @jasperla's OpenBSD gosigar PR was merged. What is the status of gosigar and topbeat on FreeBSD (and telegraf)? Can I be of assistance?

@loganbest The way I understand it, once a new port is added into the official FreeBSD ports tree, it will be automatically built and be available as a binary pkg distribution.

[girgen]

Hi!

Filebeat and packetbeat where committed to the ports tree on May 27. Topbeat didn't work well enough at the time.

[gregoryo2014]

Ah, excellent! I should have checked there before asking. Indeed it seems topbeat was less ready. I hope it is making progress.

[ruflin]

A note related to topbeat: In the 5.0.0 release Topbeat will be removed and Metricbeat is released as a replacement. Metricbeat has the same functionality in the system module and more is added. So perhaps it makes more sense to directly go with Metricbeat now?

[gregoryo2014]

Okay, that sounds a reasonable approach - how might we get the wheels in motion for Metricbeat to become a FreeBSD package? I can't find it anywhere other than elastic.co

[ruflin]

@gregoryo2014 What do you mean by "anywhere other than elastic.co"? Where did you expect to find it?

[gregoryo2014]

You've prompted me to look further, and unsurprisingly I found it in this repo. I have built metricbeat on my FreeBSD machine and it works, albeit with an error:

$ ./metricbeat 
Exiting: 4 errors: metricset 'system/cpu' is not registered, metricset not found; metricset 'system/filesystem' is not registered, metricset not found; metricset 'system/memory' is not registered, metricset not found; metricset 'system/process' is not registered, metricset not found

I started with the default metricbeat.yml which had 5 metricsets enabled, and the one it didn't complain about was system/network. I reconfigured to only enable that one, and I am now ingesting network data through logstash. Excellent.

In fact, of the 8 listed in the doco, 6 have the above complaint, and 2 work: system/diskio, system/network. I downloaded the binary from nightlies (after remembering it was there!) and got the same behaviour.

I'd like to understand why the others aren't 'registered'. Is this related to gosigar support? I see there is a directory for each one under https://github.com/elastic/beats/tree/master/metricbeat/module/system but without assistance, I'm a bit lost wandering through the source code.

[ruflin]

The reason they are not "registred" is because of the compile flags set in the golang file. For example this one compiles on freebsd, linux and windows: https://github.com/elastic/beats/blob/master/metricbeat/module/system/diskio/diskio.go#L1, but this one on https://github.com/elastic/beats/blob/master/metricbeat/module/system/memory/memory.go#L1 on darwin linux openbsd windows. Only if a metricset is compiled, it registers because of the init method: https://github.com/elastic/beats/blob/master/metricbeat/module/system/memory/memory.go#L13

It could be that some of the modules actually compile on more platforms but that are the ones we tested and should work. Best is to modify the headers and try out what happens in your case, but it means you must compile it yourself. If you have the repo in your GOPATH, just run make inside the metricbeat directory and you should get the binary.

[gregoryo2014]

Okay, I understand all of that. Sadly, I get errors relating to gosigar with any one of each of the 6 metricbeat system modules enabled as you suggested:

~/src/github.com/elastic/beats/metricbeat]$ gmake
../vendor/github.com/elastic/gosigar/concrete_sigar.go:20: cpuUsage.Get undefined (type Cpu has no field or method Get)
../vendor/github.com/elastic/gosigar/concrete_sigar.go:30: cpuUsage.Get undefined (type Cpu has no field or method Get)
../vendor/github.com/elastic/gosigar/concrete_sigar.go:49: l.Get undefined (type LoadAverage has no field or method Get)
../vendor/github.com/elastic/gosigar/concrete_sigar.go:55: m.Get undefined (type Mem has no field or method Get)
../vendor/github.com/elastic/gosigar/concrete_sigar.go:61: s.Get undefined (type Swap has no field or method Get)
../vendor/github.com/elastic/gosigar/concrete_sigar.go:73: fd.Get undefined (type FDUsage has no field or method Get)
../vendor/github.com/elastic/gosigar/sigar_unix.go:21: cannot use stat.Ffree (type int64) as type uint64 in assignment

So I cloned https://github.com/elastic/gosigar.git and got the same errors:

~/src/github.com/elastic/gosigar/examples/ps]$ go build
# github.com/elastic/gosigar
../../concrete_sigar.go:20: cpuUsage.Get undefined (type Cpu has no field or method Get)
../../concrete_sigar.go:30: cpuUsage.Get undefined (type Cpu has no field or method Get)
../../concrete_sigar.go:49: l.Get undefined (type LoadAverage has no field or method Get)
../../concrete_sigar.go:55: m.Get undefined (type Mem has no field or method Get)
../../concrete_sigar.go:61: s.Get undefined (type Swap has no field or method Get)
../../concrete_sigar.go:73: fd.Get undefined (type FDUsage has no field or method Get)
../../sigar_unix.go:21: cannot use stat.Ffree (type int64) as type uint64 in assignment

Is this because https://github.com/elastic/gosigar/pull/33 is not yet merged? From what I'm reading in the changes, those issues look like they're handled for FreeBSD.

[ruflin]

@gregoryo2014 Based on the errors, I would assume https://github.com/elastic/gosigar/pull/33 will not solve all the errors above but I'm hopefully wrong. Could you also fetch the most recent version of master as we had a platform related fix recently?

What you can try is to fetch the gosigar version from the pull request, put it into your gopath and remove the gosigar version under vendor. Then you will see if it works.

@andrewkroh Do you know some more details here?

[gregoryo2014]

Beyond simple clones of entire repos, git baffles me. I managed the following, which is hopefully helpful:

~/src/github.com/knz]$ git clone https://github.com/knz/gosigar.git
~/src/github.com/knz/gosigar/examples/ps]$ go build
# github.com/elastic/gosigar
../../../../elastic/gosigar/concrete_sigar.go:20: cpuUsage.Get undefined (type Cpu has no field or method Get)
../../../../elastic/gosigar/concrete_sigar.go:30: cpuUsage.Get undefined (type Cpu has no field or method Get)
../../../../elastic/gosigar/concrete_sigar.go:49: l.Get undefined (type LoadAverage has no field or method Get)
../../../../elastic/gosigar/concrete_sigar.go:55: m.Get undefined (type Mem has no field or method Get)
../../../../elastic/gosigar/concrete_sigar.go:61: s.Get undefined (type Swap has no field or method Get)
../../../../elastic/gosigar/concrete_sigar.go:73: fd.Get undefined (type FDUsage has no field or method Get)
../../../../elastic/gosigar/sigar_unix.go:21: cannot use stat.Ffree (type int64) as type uint64 in assignment

I think that means gosigar from https://github.com/knz/gosigar and therefore @knz's PR to https://github.com/elastic/gosigar isn't working.

In case it is of use, I did this too:

~/src/github.com/knz/gosigar]$ export GOPATH=`pwd`:~
~/src/github.com/elastic/beats/vendor/github.com/elastic]$ rm -r gosigar/
~/src/github.com/elastic/beats/vendor/github.com/elastic]$ cp -Rp ~/src/github.com/knz/gosigar gosigar
~/src/github.com/elastic/beats/metricbeat]$ gmake
go build
# github.com/elastic/beats/vendor/github.com/elastic/gosigar
cc: warning: argument unused during compilation: '-pthread'
# github.com/elastic/beats/vendor/github.com/elastic/gosigar
../vendor/github.com/elastic/gosigar/sigar_freebsd.go:110: readFile redeclared in this block
    previous declaration at ../vendor/github.com/elastic/gosigar/sigar_linux_common.go:344
gmake: *** [../libbeat/scripts/Makefile:62: build] Error 2

[ruflin]

Not sure if you hit more a Golang then a git issue. If you work on a clone of the gosigar project, it must be still in the github.com/elastic/gosigar namespace to work (not the knz) one. Otherwise you would have to change all package references. The easiest way here is to just clone it to github.com/elastic/gosigar and use different remotes if you want to work with both versions.

[ruflin]

@gregoryo2014 In any case, looking at the code directly and your output I think you definitively spotted a problem with the readFile method as it is defined twice. @knz Did it build on your machine under freebsd?

[ruflin]

@andrewkroh Unfortunately our freebsd build shows something similar to the error that @gregoryo2014 reported. Seems like we broke the build some time ago :-( http://build-eu-00.elastic.co/view/Beats/job/beats-freebsd/411/console

[knz]

Please check https://github.com/elastic/gosigar/pull/34.

[gregoryo2014]

It builds and works! I am now ingesting data from all 8 system modules.

[gregoryo2014]

Now I'm confused. https://github.com/elastic/beats/pull/1966 is merged, but there are now two issues:

1. No FreeBSD nightly builds since July 6th, and no builds at all since July 8th. Is there a new location for nightlies?
2. None of the builds seem to use a gosigar with the freebsd tags enabled - they all fail on the 6 modules enabled in https://github.com/elastic/beats/pull/1966

[andrewkroh]

@gregoryo2014 I can explain.

The build you linked to is created by Jenkins (a linux slave). It is not capable of cross-compiling Metricbeat for FreeBSD after the introduction of C code specifically for FreeBSD because this would required a FreeBSD cross-compiler which is not setup there. Previously it worked because there was no FreeBSD-specific C code (there was no cgo usage).

We need to add FreeBSD support to our packaging code to have it generate a tar.gz for FreeBSD. This would be published to https://beats-nightlies.s3.amazonaws.com/index.html?prefix=metricbeat/ with our other snapshot releases. In order to make this happen we need to update the scripts here. Basically we have the cross-compilers installed inside of a container and run the go build inside of the container, then tar up the resulting binary + config files, and release. If anyone wants to help work this... 👍

[gregoryo2014]

While I've had no time for real work on this to date, and not likely any time soon, I'd like to report that I had a good experience building the tools again myself tonight. I simply cloned the repository, installed Go, and typed gmake - this in both the metricbeat and filebeat subdirectories, and now I'm happily using those binaries.

Out of interest I built libbeat, packetbeat and winlogbeat (!) while I was there, and they all built successfully.

[ruflin]

@gregoryo2014 Thanks for reporting back. @andrewkroh This makes me wonder if we should close this issue?

[loganbest]

@ruflin I would say give it some more testing across multiple versions of bsd before closing just to make sure. Or we can close it and make those additional bug reports should there be any.

[msimerson]

+1 for closing the issue. Leaving it open leaves one with the impression that FreeBSD support is not done, whereas it's been working for many months.

[ph]

@msimerson I'll close this following your words.