csv-values: Pipes in data are seen as delimiter between check output and performance data

slalomsk8er commented 7 months ago

This issue respects the following points:

[X] This is a bug, not a question or a setup/configuration issue.
[X] This issue is not already reported on Github (I've searched it).
[X] I use the latest release of the Monitoring Plugins (https://github.com/Linuxfabrik/monitoring-plugins/releases).
[X] I agree to follow Monitoring Plugins's Code of Conduct.

Which variant of the Monitoring Plugins do you use?

[ ] .rpm/.deb package from repo.linuxfabrik.ch
[ ] Compiled for Linux (.tar/.zip from download.linuxfabrik.ch)
[X] Compiled for Windows (from download.linuxfabrik.ch)
[ ] Source Code from GitHub

Bug description

While setting up a check to monitor long running LDAP queries on the domain controllers I discovered the following behavior:

All lines were cut after the first |

I tried to mitigate with replace( Filter, "|", "¦" ) in the --warning-query and now the results aren't cut any longer but the perfdata is still mangled ) ) ! 3305761 ! 24 ! INTERSECT_INDEX:34754:I;|'cnt_warn'

Steps to reproduce - Plugin call

'C:\ProgramData\icinga2\usr\lib64\nagios\plugins\csv-values.exe' '--chunksize' '1000' '--columns-query' 'LDAPServer TEXT,TimeGenerated TEXT,ClientIP TEXT,ClientPort INTEGER,StartingNode TEXT,Filter TEXT,SearchScope TEXT,AttributeSelection TEXT,ServerControls TEXT,VisitedEntries INTEGER,ReturnedEntries INTEGER,UsedIndexes TEXT,PagesReferenced TEXT,PagesReadFromDisk INTEGER,PagesPreReadFromDisk INTEGER,CleanPagesModified INTEGER,DirtyPagesModified INTEGER,SearchTimeMS INTEGER,AttributesPreventingOptimization TEXT' '--delimiter' ',' '--filename' 'D:\Downloads\ICTADC22WP01\1644-ictadc22wp01_20231201-0159_DirectoryService.csv' '--quotechar' '"' '--timeout' '3' '--warning' '0' '--warning-query' 'select TimeGenerated,ClientIP,SearchTimeMS,StartingNode,replace( Filter, "|", "¦" ),VisitedEntries,ReturnedEntries,UsedIndexes from data where SearchTimeMS >=20000 order by SearchTimeMS DESC'

Steps to reproduce - Data

get some CSV with |s in the data
1. generate eventlogs according to https://learn.microsoft.com/en-US/troubleshoot/windows-server/identity/event1644reader-analyze-ldap-query-performance
2. use the part in Event1644Reader.ps1 before the excel generation the CSV files
3. or use or craft any other CSV with | in the data
setup a csv-values based service

Environment

OS Name Microsoft Windows Server 2022 Standard Version 10.0.20348 Build 20348

Plugin Version

csv-values.exe: v2023051201 by Linuxfabrik GmbH, Zurich/Switzerland

Python version

No response

List of Python modules

No response

Additional Information

No response

slalomsk8er commented 3 months ago

Also happens for URIs in php-fpm-status messages:

Plugin-Ausgabe
Pool www (dynamic): 2x max children reached [WARNING], 1.6M connections, 35 processes (2 active, 33 idle), Up 1W 6D (since 2024-03-07 04:11:25)
PID     ! Reqs ! ReqDur      ! Request URI                                                                                                                      ! POST ! AuthUser 
--------+------+-------------+----------------------------------------------------------------------------------------------------------------------------------+------+----------
1021555 ! 4497 ! 745ms 325us ! /icingaweb2/index.php?(service=%2Apassive%20-%20robot%2A
...
Performancedaten
  | Bezeichnung | Wert | Max | Warnung | Kritisch
-- | -- | -- | -- | -- | --
  | service_display_name | - | - | - | -
  | ! - ! - service | - | - | - | -
  | ! - ! - hostgroup_alias | - | - | - | -
  | ! - ! - servicegroup_alias | - | - | - | -
  | max children reached | 2.00 | - | 1.00 | 100.00
  | idle processes | 30.00 | 35.00 | - | -
  | active processes | 5.00 | 35.00 | - | -
  | ! - ! -\| | 1,647,676.00 c | - | - | -
  | listen queue | 0.00 | 0.00 | - | -
  | listen queue len | 0.00 | - | - | -
  | slow requests | 0.00 | - | 1.00 | 100.00
  | start since | 13,24 d | - | - | -
  | queue usage | 0% | - | 80% | 90%

NavidSassan commented 3 months ago

maybe add escaping / replacing in lib.base.oao?

markuslf commented 3 months ago

I'll check.

markuslf commented 3 months ago

Fixed in https://github.com/Linuxfabrik/lib/commit/c9072e3a537054e0c5b57899a17385f4bb860698

Linuxfabrik / monitoring-plugins