Upgrade Datadog Buildpack to Version 4.6.0

[hadirgax]

Describe the Enhancement

I'm requesting an upgrade of the Datadog Buildpack from its current version 3.6.0 to the latest version 4.6.0. This upgrade will provide several benefits, including:

• Access to new features and enhancements: Version 4.6.0 introduces new capabilities and improvements that can enhance visibility and monitoring for NodeJS applications.
• Compatibility with latest Datadog Agent: Using the latest buildpack ensures compatibility with the newest Datadog Agent, ensuring seamless integration and access to the most up-to-date features.
• Security updates: The upgrade addresses potential vulnerabilities and incorporates security patches related to GoLang version, reinforcing the overall security of NodeJS applications.

Possible Solution

To implement this change, the following steps can be taken:

1. Update the Datadog Buildpack dependency in the buildpack.toml and package.toml files to version 4.6.0.
2. Thoroughly test the updated buildpack to ensure compatibility and functionality with NodeJS applications.
3. Update any relevant documentation to reflect the new buildpack version and its usage.

Motivation

The primary motivation for this request is to:

• Benefit from the latest Datadog features and enhancements: Gain access to new monitoring capabilities and improvements offered in version 4.6.0.
• Maintain compatibility with the latest Datadog Agent: Ensure seamless integration and avoid potential issues arising from version mismatches.
• Enhance security: Address potential vulnerabilities and reinforce security measures by using the latest buildpack.
• Stay up-to-date with best practices: Adhere to recommended practices by using the latest supported buildpack version.

[dmikusa]

@hadirgax & @gitricko - Hi, thanks for reaching out here. I'm hoping to get a little bit of detail from you on your usage of the Datadog buildpack w/Node.js.

The way the Datadog, and our other APM buildpacks, work with Node.js is that they basically run an npm install of the agent libraries and then modify your app's main JS file to inject the agent code.

I'm hoping to get some feedback from you on this approach, and if you've considered installing the APM module directly and adding it to the source code of your project directly also (essentially the out-of-the-box approach for your APM tool of choice).

The approach that the buildpack takes currently has a number of drawbacks for Node.js integration:

• It's modifying the installed modules at build-time so that might be less visible for auditing purposes (although it should show up in the SBOM generated for the image, or if you scan the produced image). It's not in your source code or source repository though, so if you're auditing against the code you won't see it.
• It has to modify the application source code. This can go wrong in a number of ways, so it's pretty fragile and error-prone.
• It does the bare minimum for integration, so if you want additional customization or optimization of how the agent is integrated then you end up needing to install it directly anyway.
• It injects a require(..) to load the agent, which is an older way of loading modules. This might not play nice with all apps.
• It requires the buildpack maintainers to bump releases and you lose that control, which is I believe the point of the issue you opened.

Anyway, if you have a moment and can send some feedback on your use case for using this feature & the benefits you get from using it this way, it would be much appreciated.

tl;dr - I'm considering an RFC to deprecate adding the Node.js APM agents directly, but before that, I'd like to gather feedback from people using it.

Thanks!

[mhdawson]

@hadirgax, @gitricko , @qard, @bengl it would be good to get some contenxt/help.

I had to back out the update to the 5.x line of the DataDog package as it failed the existing tests.