search

elastic / integrations

Elastic Integrations
https://www.elastic.co/integrations
Other
187 stars 392 forks source link

[Enhancement/Fix] - F5 BIG-IP Azure hosted (Virtual Edition) - N/A values #9275

Open gbocchini opened 4 months ago

gbocchini commented 4 months ago

In BIG-IP 15.1.10.2 Build 0.44.2 Engineering Hotfix available for example in Azure as BigIP Virtual Edition some fields are coming filled with N/A string (using the defaults of the software, going against https://github.com/elastic/integrations/blob/main/packages/f5_bigip/data_stream/log/_dev/test/pipeline/test-pipeline-bigip-asm.log) but the pipeline shipped with the integration doesn't cover this condition, only checks for empty/null value, which makes the docs being ingested with N/As, instead of not filling it.

A simple complement of the conditions checks solves the issue.

Example: instead of only ctx.json?.management_ip_address_2 != '' we could have ctx.json?.management_ip_address_2 != '' || ctx.json?.management_ip_address_2 == "N/A"

Oddly commented 2 weeks ago

Instead of a conditional per task, wouldn't it be easier to just delete all the fields that contain 'N/A' at the start of the general pipeline? It looks like the change to use "N/A" as value instead of empty fields, if a value is empty in ASM logging format is permanent and will affect all ASM versions > 15. source

We use on prem ASM > 15.0.0 and at the moment, the integration is unusable. The pipeline is generating errors on trying to convert IP's with value "N/A", so the data is useless.