search

influxdata / telegraf

Agent for collecting, processing, aggregating, and writing metrics, logs, and other arbitrary data.
https://influxdata.com/telegraf
MIT License
14.59k stars 5.56k forks source link

Freebsd ARM support #5282

Closed SomKen closed 5 years ago

SomKen commented 5 years ago

Feature Request

Any chance of a Freebsd arm build for Telegraf? With Netgate starting to release more ARM based PFSense devices, telegraf is the last feature I need to fully switch off x86.

I know the Freebsd ports tree is normally pretty good, but there's currently no support for telegraf on ARM. As telegraf is more than happy to run on Freebsd and Linux ARM, it seems like there's a build we're just missing.

Maybe this would be better to contact the maintainer of the telegraf builds on freebsd, but I'd trust influx more.

Proposal:

Currently, pfsense, and mainly freebsd, are missing a telegraf package on ARMv7.

Current behavior:

No freebsd ARM package.

Desired behavior:

A freebsd ARM package.

Use case: [Why is this important (helps with prioritizing requests)]

PFSense on ARM

danielnelson commented 5 years ago

I be happy to accept a patch to scripts/build.py, I might work on this in the future but won't have time in the short term.

@girgen @W0CHP have done FreeBSD packaging of Telegraf, have you considered providing packages for arm?

W0CHP commented 5 years ago

I’ll take a crack at it this week; I’m running a FreeBSD 11.2-p8 build environment on an ARM box, as well as a dev PFsense instance on ARM.

If successful, I’ll likely still submit my work to the upstream FreeBSD Ports tree for approval/inclusion as well.

Thanks for the mention, @danielnelson!

SomKen commented 5 years ago

With FreeBSD, you're able to build ARM packages on x86.

From a random guy on the internet, it seems like an environment variable you set prior to building. YMMV. I'd do this, but I don't have freebsd setup directly.

https://www.reddit.com/r/PFSENSE/comments/8ufp0x/cant_find_telegraf_package_sg3100_pfsense_243/e1qu0ps

At one point, pfsense did support telegraf, but Golang wouldn't compile anymore. That doesn't seem to be a problem now. https://github.com/golang/go/wiki/GoArm

W0CHP commented 5 years ago

I tried doing a quick build, but it's failing at the gopsutil plugin (output at end).

Both cross-compiling, and compiling directly on an armv7 FreeBSD machine, fail when building gopsutil.

Since telegraf depends on gopsutil, I recommend, @SomKen, that perhaps you can file an issue in @shirou's shirou/gopsutil project. I'm afraid I can't perform the build, until there is full armv7 (FreeBSD) support from that dependency.

armv7 FreeBSD gopsutil build output:

github.com/shirou/gopsutil/cpu ../../shirou/gopsutil/cpu/cpu_freebsd.go:25:16: undefined: cpuTimes ../../shirou/gopsutil/cpu/cpu_freebsd.go:42:31: undefined: cpuTimes ../../shirou/gopsutil/cpu/cpu_freebsd.go:66:38: undefined: cpuTimes ../../shirou/gopsutil/cpu/cpu_freebsd.go:72:15: undefined: cpuTimes ../../shirou/gopsutil/cpu/cpu_freebsd.go:87:13: undefined: cpuTimes github.com/shirou/gopsutil/disk ../../shirou/gopsutil/disk/disk_freebsd.go:26:36: undefined: MNT_WAIT ../../shirou/gopsutil/disk/disk_freebsd.go:31:15: undefined: Statfs ../../shirou/gopsutil/disk/disk_freebsd.go:32:28: undefined: MNT_WAIT ../../shirou/gopsutil/disk/disk_freebsd.go:38:17: undefined: MNT_RDONLY ../../shirou/gopsutil/disk/disk_freebsd.go:41:17: undefined: MNT_SYNCHRONOUS ../../shirou/gopsutil/disk/disk_freebsd.go:44:17: undefined: MNT_NOEXEC ../../shirou/gopsutil/disk/disk_freebsd.go:154:9: undefined: Bintime ../../shirou/gopsutil/disk/disk_freebsd.go:163:22: undefined: Statfs ../../shirou/gopsutil/disk/disk_freebsd.go:167:54: undefined: Statfs ../../shirou/gopsutil/disk/disk_freebsd.go:182:32: undefined: Devstat ../../shirou/gopsutil/disk/disk_freebsd.go:44:17: too many errors

danielnelson commented 5 years ago

Here is an issue on the gopsutil issue tracker: https://github.com/shirou/gopsutil/issues/627

W0CHP commented 5 years ago

Awesome I’ll try the trick referenced in the issue!

W0CHP commented 5 years ago

OK I got it built and working on arm6 and arm7 FreeBSD-11.2-p8.

Note that I used/built Telegraf v1.9.0, which is the current version in the official FreeBSD Ports tree (for now).

As @danielnelson mentioned, scripts/build.py needed to be slightly modified. Here is the (v1.9.0) patch for that: https://raw.githubusercontent.com/W0CHP/FreeBSD-ARM/master/net-mgmt/telegraf/files/patch-scripts_build_py

The port build Makefile has been tweaked to address shirou/gopsutil#627

Test Port tree if one wants to build the port: https://github.com/W0CHP/FreeBSD-ARM/tree/master/net-mgmt/telegraf

FreeBSD-armv7 binary package* for folks that can't/won't build: https://github.com/W0CHP/FreeBSD-ARM/raw/master/net-mgmt/telegraf/pkg/telegraf-1.9.0.txz

Here's a screenshot of data being collected on a Telegraf-1.9.0 instance on FreeBSD-11.2 armv7: https://raw.githubusercontent.com/W0CHP/FreeBSD-ARM/master/net-mgmt/telegraf/Telegraf-ARM.png

*Test package; use at your own risk.

SomKen commented 5 years ago

Quick turn around @W0CHP! Thanks for the inital work.

I'll get this tested shortly on pfsense to make sure its happy too. At least this is a start.

girgen commented 5 years ago

Cool @W0CHP ! If you send me the patch for the telegraf port, I can commit the fix. Either here, directly to me or as a bugzilla PR.

SomKen commented 5 years ago

@W0CHP for the record, your build is working on my Netgate SG3100 running PFSense 2.4.4-p2.

W0CHP commented 5 years ago

@girgen It looks like you already merged my patches and committed(?) :-) https://github.com/freebsd/freebsd-ports/commit/8a143a3599a11c8ddd10f9e82b587bd696ac4d32

Thanks!

W0CHP commented 5 years ago

@danielnelson FYI all of these changes have been merged and committed upstream to the FreeBSD Ports tree (thanks @girgen!).

girgen commented 5 years ago

@girgen It looks like you already merged my patches and committed(?) :-) freebsd/freebsd-ports@8a143a3

Thanks!

Yeah, I think I found all changes (the patch file, adding ARCH and the two cp lines), so I committed it together with an update to 1.9.2. Hope I didn't miss anything? :-)

SomKen commented 5 years ago

With the code moving to the freebsd tree, should I close this?

W0CHP commented 5 years ago

Yeah, I think I found all changes (the patch file, adding ARCH and the two cp lines), so I committed it together with an update to 1.9.2. Hope I didn't miss anything? :-)

You nailed it, @girgen! And I tested on two ARM installs from the updated ports tree. Works great.

With the code moving to the freebsd tree, should I close this?

@SomKen I think closing this is appropriate. Thanks for issuing the FR; and thanks again @girgen for the commit. :-)

SomKen commented 5 years ago

Thanks again!

danielnelson commented 5 years ago

Let's leave this open until fixed in gopsutil.

girgen commented 5 years ago

I think both the patch and the copies of files should propagate upstreams from the port, i.e. it should be committed here and in the gopsutil trees.

I guess a pull request is preferred? @W0CHP should get the glory for fixing it. :)

danielnelson commented 5 years ago

We don't vendor, so it would only go upstream, here is the link again to the upstream issue: https://github.com/shirou/gopsutil/issues/627.

danielnelson commented 5 years ago

I believe this was addressed by https://github.com/shirou/gopsutil/pull/632

SomKen commented 3 years ago

@W0CHP You around? Can you share me the repo you had for building this. Netgate still hasn't released Telegraf after two years....

W0CHP commented 3 years ago

@W0CHP You around? Can you share me the repo you had for building this. Netgate still hasn't released Telegraf after two years....

I'm around! I've moved the repo here: https://repo.w0chp.net/Chipster/FreeBSD-Ports-Custom/src/master/ports/net-mgmt/telegraf

I have not maintained it in two years due to time constraints. Hope that helps.

SomKen commented 3 years ago

Got it! Thanks. It's insane this never got solved, but I think you did all the heavy lifting needed. I owe you a beer.

SomKen commented 3 years ago

@W0CHP You around? Can you share me the repo you had for building this. Netgate still hasn't released Telegraf after two years....

I'm around! I've moved the repo here: https://repo.w0chp.net/Chipster/FreeBSD-Ports-Custom/src/master/ports/net-mgmt/telegraf

I have not maintained it in two years due to time constraints. Hope that helps.

Go cross compiles.... buahahahah

SomKen commented 3 years ago

@W0CHP thanks again for your help before. This story has come to an end... https://github.com/influxdata/telegraf/commit/279dda21ef6163fdce1b9365fc8931cb8213d474

W0CHP commented 3 years ago

@W0CHP thanks again for your help before. This story has come to an end... 279dda2

My pleasure, and nice work!!!

SomKen commented 3 years ago

https://docs.influxdata.com/telegraf/v1.18/about_the_project/release-notes-changelog/

"Add FreeBSD ARMv7 build." Front and center...