search

GSA-TTS / cg-logshipper

Sample code for draining Cloud Foundry logs to S3 and New Relic using FluentBit
Other
7 stars 2 forks source link

Update New Relic plugin to version 2.0.0 #17

Closed akf closed 4 months ago

akf commented 4 months ago

Updates the New Relic output plugin to version 2.0.0: https://github.com/newrelic/newrelic-fluent-bit-output/releases/tag/v2.0.0

There is also an incidental update to the TODO list in the README.

I tested this by deploying this app first to my sandbox, and then to a space with restricted egress and a proxy, to verify that the new version of newrelic-fluent-bit-output successfully sends logs to New Relic both without and with a proxy server (PROXYROUTE env set). The manifest for the test had variant names for the app itself and the services, and the fluent.bit.conf file used the dummy input with multiple samples, plus the stdout OUTPUT because I wanted to see the logs.

Note that although the timing of this PR was prompted by a vulnerability alert from New Relic (NR) and that NR says "bumping fluent bit version to 3.0.4," the plugin itself is largely independent of the fluent-bit version (probably thanks to fluent bit's backwards compatibility policy). The newrelic-fluent-bit-output repo (and src downloads) include Docker images that pull in fluent bit version 3.0.4, which could be used standalone to build a log shipper. We're installing fluent-bit via apt and getting the latest packaged version every time we start or restart an app.

mogul commented 4 months ago

To think about for future: How can we grab the binary just-in-time rather than keeping a copy of that file here in the repo? 🤔

akf commented 4 months ago

To think about for future: How can we grab the binary just-in-time rather than keeping a copy of that file here in the repo? 🤔

Just to clarify -- we want to grab the binary for a specific release of the New Relic output plugin that we've already tested, not the latest version, right? I could also see adding a build script that took a parameter specifying what version of the plugin to download, with a reasonable fallback default.

Or we could keep a copy of that binary in an artifact store TBD ...

mogul commented 4 months ago

Yes, I meant just ensuring that we don't have the binary artifact in the repository. I'm thinking of that recent case where a contributor committed an opaque binary blob as a test fixture, which later turned out to be a supply-chain attack.

asteel-gsa commented 4 months ago

Yup, agreed. We can probably get away with something similar to this in a startup script to curl the binary we need, just so we dont keep it in the repo.