[Security Integrations] Deprecating RSA2ELK Filebeat Modules

[jamiehynds]

Several Filebeat modules which were originally converted from open source RSA parsers, are still under technical preview. Many of these modules have been rewritten as Elastic Agent integrations. These modules should be deprecated on the Filebeat side, to avoid users running sub-par, technical preview integrations, especially where better options exist via agent integrations.

To discuss: how does deprecation work? Is it similar to agent integrations where we can add a deprecation notice to each module? Can we remove this modules from future Stack releases? If we stop shipping these modules, will existing users of these modules still be able to use them?

Modules to deprecate:

• Barracuda
• Bluecoat
• Cisco Nexus (fileset within Cisco module)
• Cisco Meraki (as above)
• Cylance
• F5
• Fortinet Client Endpoint (fileset within Fortinet)
• Fortinet Fortimail (as above)
• Fortimanager (as above)
• Imperva
• Infoblox
• Juniper Junos
• Juniper Netscreen
• Microsoft DHCP
• Netscout
• Proofpoint
• Radware
• Snort
• Sonicwall
• Sophos UTM
• Squid
• Tomcat
• Zscaler

[elasticmachine]

Pinging @elastic/security-external-integrations (Team:Security-External Integrations)

[taylor-swanson]

Since I recently handled an SDH involving one of these modules (sonicwall), I have an interest in working on this.

There's an existing example of a deprecated module, misp. It has a notice in its readme declaring that the module is deprecated:

deprecated::[7.14.0,"This module is deprecated. Use the <<filebeat-module-threatintel,Threat Intel module>> instead."]

https://github.com/elastic/beats/blob/main/x-pack/filebeat/module/misp/_meta/docs.asciidoc?plain=1#L8

It points to a different module as a way forward, but in other cases, it could point to a Fleet package (this is the case with sonicwall).

In most cases, the deprecated module has been superseded by a Fleet package. @jamiehynds, I haven't looked closely at this list yet, but I imagine we want to point users towards the Fleet package? Would there be any cases where an existing, non-deprecated module would be an option (similar to what is seen in the misp package)?

[taylor-swanson]

Here's a summary for the paths forward for the packages:

• Barracuda
  • Superseded by barracuda Fleet package
• Bluecoat
  • ⚠️ No Path Forward (bluecoat Fleet package has also been deprecated)
• Cisco Nexus
  • Superseded by cisco_nexus Fleet package (⚠️ in technical preview)
• Cisco Meraki
  • Superseded by cisco_meraki Fleet package
• Cylance
  • ⚠️ No Path Forward (cylance Fleet package is an rsa2elk package)
• F5
  • Superseded by f5-bigip Fleet package
• Fortinet Client Endpoint
  • Superseded by fortinet_forticlient Fleet package
• Fortinet Fortimail
  • Superseded by fortinet_fortimail Fleet package
• Fortimanager
  • Superseded by fortinet_fortimanager Fleet package
• Imperva
  • ⚠️ No Path Forward currently, but improvements to imperva Fleet package (which is an rsa2elk package) is in the works
• Infoblox
  • Superseded by infoblox_nios Fleet package
• Juniper Junos
  • Superseded by juniper_srx Fleet package
• Juniper Netscreen
  • ⚠️ No Path Forward (Juniper has announced EOL of Netscreen appliances)
• Microsoft DHCP
  • Superseded by microsoft_dhcp Fleet package
• Netscout
  • ⚠️ No Path Forward (netscout Fleet package is an rsa2elk package)
• Proofpoint
  • Supeseded by proofpoint_tap Fleet package
• Radware
  • ⚠️ No Path Forward (radware Fleet package is an rsa2elk package)
• Snort
  • Superseded by snort Fleet package
• Sonicwall
  • Superseded by sonicwall Fleet package
• Sophos UTM
  • Superseded by sophos Fleet package
• Squid
  • ⚠️ No Path Forward (squid Fleet package is an rsa2elk package)
• Tomcat
  • Superseded by apache_tomcat Fleet package
• Zscaler
  • Superseded by zscaler_zia Fleet package

[jamiehynds]

Thanks for working through this @taylor-swanson - totally agree on pointing users to our agent integrations where applicable. In cases where there isn't an integration, we can recommend one of the Custom packages to at least ingest the data, and raise an issue in our integrations repo to request support for an integration.

List above looks good, just some minor tweaks.

• Imperva: we're in the middle of building an Imperva integration which will be available as an update to the current RSA2ELK package. No path forward today, but there will be very soon.

• Proofpoint: we're ok to recommend Proofpoint TAP as the integration.

• Tomcat - the o11y have built a new integration for Tomcat which we can recommend. https://github.com/elastic/integrations/tree/main/packages/apache_tomcat

[taylor-swanson]

Thanks, @jamiehynds! I updated the list above accordingly.

[ebeahan]

totally agree on pointing users to our agent integrations where applicable. In cases where there isn't an integration, we can recommend one of the Custom packages to at least ingest the data, and raise an issue in our integrations repo to request support for an integration.

@jamiehynds - @taylor-swanson and I were discussing, and @taylor-swanson shared the idea of a dedicated "deprecated integration" page somewhere in our docs. The page would provide a single place capturing the guidance you listed here (direct users how to use the custom package to replace a deprecated integration, direct users to open an issue in the integrations repo, etc.). Deprecated integrations could then link to it.

WDYT?

[andrewkroh]

@jamiehynds @taylor-swanson Is there a timeline for full removal of the deprecated modules? I want to mark my calendar ~so I can celebrate~ so we don't forget. 📆

[jamiehynds]

@jamiehynds @taylor-swanson Is there a timeline for full removal of the deprecated modules? I want to mark my calendar ~so I can celebrate~ so we don't forget. 📆

I'd suggest we aim for 8.14 for full removal, so that gives customers ~6 months notice.