toni-moreno / snmpcollector

A full featured Generic SNMP data collector with Web Administration Interface for InfluxDB
MIT License
286 stars 53 forks source link

Force returned values as signed int32? #513

Closed matejv closed 2 years ago

matejv commented 2 years ago

Hello

I have a device, which returns all data as Gauge32. There is an OID that should be interpreted as an signed 32bit integer.

$ snmpwalk -v2c -c comm device .1.3.6.1.4.1.476.1.42.3.9.30.1.20.1.2.1.5266.2
SNMPv2-SMI::enterprises.476.1.42.3.9.30.1.20.1.2.1.5266.2 = Gauge32: 4294967295

The value 4294967295 is actually 2^32 or if taken as a signed int32 it will be -1. -1 is the value I need.

When I set-up SNMP Metric, I selected for DataSrcType all possible integer values. No matter what I select I always get 4294967295 as a result by snmpcollector.

Does DataSrcType not force casting values as selected type? It seems to me that snmpcollector respects what type the data is as returned via SNMP no matter what (so gauge32 in my case).

Debug for problematic device log bellow:

time="2021-12-21 16:11:00" level=info msg="Init gather cycle mode" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="-------Processing measurement : ahv_metrics" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="LEN 2 : [.1.3.6.1.4.1.476.1.42.3.9.30.1.20.1.2.1.5266.2 .1.3.6.1.4.1.476.1.42.3.9.30.1.20.1.2.1.5266.1] | client : &{snmpClient:0xc0000f2280 Log:0xc0002883c0 ID:mydevice-ahv_metrics DisableBulk:true Connected:true ConnectionParams:{Host:mydevice Port:161 Timeout:5 Retries:0 SnmpVersion:2c Community:mycomm MaxRepetitions:50 MaxOids:60 Debug:true V3Params:{SecLevel: AuthUser: AuthPass: PrivPass: PrivProt: AuthProt: ContextName: ContextEngineID:}}}" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="Getting snmp data from 0 to 2" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="DEBUG pdu [{Name:.1.3.6.1.4.1.476.1.42.3.9.30.1.20.1.2.1.5266.2 Type:Gauge32 Value:4294967295}] || Value type uint [47617567653332] " device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="DEBUG pdu [{Name:.1.3.6.1.4.1.476.1.42.3.9.30.1.20.1.2.1.5266.2 Type:Gauge32 Value:4294967295}] || Value type uint [47617567653332]" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="OK measurement .1.3.6.1.4.1.476.1.42.3.9.30.1.20.1.2.1.5266.2 SNMP RESULT OID %!s(uint=4294967295) MetricFound" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="DEBUG pdu [{Name:.1.3.6.1.4.1.476.1.42.3.9.30.1.20.1.2.1.5266.1 Type:Gauge32 Value:91}] || Value type uint [47617567653332] " device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="DEBUG pdu [{Name:.1.3.6.1.4.1.476.1.42.3.9.30.1.20.1.2.1.5266.1 Type:Gauge32 Value:91}] || Value type uint [47617567653332]" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="OK measurement .1.3.6.1.4.1.476.1.42.3.9.30.1.20.1.2.1.5266.1 SNMP RESULT OID %!s(uint=91) MetricFound" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=info msg="Not Oid CONDITIONEVAL metrics exist on this measurement" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=info msg="Not EVAL metrics exist on  this measurement" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="DEBUG METRIC  CONFIG &{ID:lgpEnvCompressor1HeadPressure FieldName:compressor_1_head_pressure_bar Description:LIEBERT-GP-FLEXIBLE-MIB::lgpFlexibleEntryUnsignedIntegerValue.1.2.1.5266.1 BaseOID:.1.3.6.1.4.1.476.1.42.3.9.30.1.20.1.2.1.5266.1 DataSrcType:INTEGER GetRate:false Scale:0 Shift:0 IsTag:false ExtraData: Conversion:1 Names:map[]}" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="generating field for compressor_1_head_pressure_bar value 91 " device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="DEBUG METRIC &{cfg:0xc0005647e0 Valid:true CookedValue:91 CurValue:<nil> LastValue:<nil> CurTime:2021-12-21 16:11:00.001629697 +0100 CET m=+788.106927957 LastTime:0001-01-01 00:00:00 +0000 UTC ElapsedTime:0 Compute:<nil> Scale:0xa00340 Convert:0xa03e20 SetRawData:0xa005c0 RealOID:.1.3.6.1.4.1.476.1.42.3.9.30.1.20.1.2.1.5266.1 Report:1 re:<nil> mm:[] expr:<nil> condflt:<nil> log:0xc0002883c0}" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="DEBUG METRIC  CONFIG &{ID:lgpEnvCompressor2HeadPressure FieldName:compressor_2_head_pressure_bar Description:LIEBERT-GP-FLEXIBLE-MIB::lgpFlexibleEntryUnsignedIntegerValue.1.2.1.5266.2 BaseOID:.1.3.6.1.4.1.476.1.42.3.9.30.1.20.1.2.1.5266.2 DataSrcType:Integer32 GetRate:false Scale:0 Shift:0 IsTag:false ExtraData: Conversion:1 Names:map[]}" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="generating field for compressor_2_head_pressure_bar value 4294967295 " device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="DEBUG METRIC &{cfg:0xc000564750 Valid:true CookedValue:4294967295 CurValue:<nil> LastValue:<nil> CurTime:2021-12-21 16:11:00.001629697 +0100 CET m=+788.106927957 LastTime:0001-01-01 00:00:00 +0000 UTC ElapsedTime:0 Compute:<nil> Scale:0xa00340 Convert:0xa03e20 SetRawData:0xa005c0 RealOID:.1.3.6.1.4.1.476.1.42.3.9.30.1.20.1.2.1.5266.2 Report:1 re:<nil> mm:[] expr:<nil> condflt:<nil> log:0xc0002883c0}" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="FIELDS:map[compressor_1_head_pressure_bar:91 compressor_2_head_pressure_bar:4294967295]" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=debug msg="GENERATED INFLUX POINT[ahv] value: ahv,fqdn=mydevice compressor_1_head_pressure_bar=91i,compressor_2_head_pressure_bar=4294967295i 1640099460001629697" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=info msg="[measurement] STATS SNMP GET: snmp polling took [0.012883 seconds] SNMP: Gets [2] , Processed [2], Errors [0]" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=info msg="[measurement] STATS SNMP FILTER: filter polling took [0.000000 seconds] " device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=info msg="[measurement] STATS INFLUX: influx send took [0.000017 seconds]" device=mydevice measurement=ahv_metrics
time="2021-12-21 16:11:00" level=info msg="MeasurementLoop new Iteration" device=mydevice measurement=ahv_metrics

I'm running version 0.12.0. I also tested on 0.10.0 with the same results.

Attaching my configuration as well: bulkexport_20211221_1614.txt

toni-moreno commented 2 years ago

Hello @matejv , I've reviewed the RFC where is defined this snmp type here ( https://datatracker.ietf.org/doc/html/rfc2578#section-7.1.7)

7.1.7.  Gauge32

   The Gauge32 type represents a non-negative integer, which may
   increase or decrease, but shall never exceed a maximum value, nor
   fall below a minimum value.  The maximum value can not be greater
   than 2^32-1 (4294967295 decimal), and the minimum value can not be
   smaller than 0.  The value of a Gauge32 has its maximum value
   whenever the information being modeled is greater than or equal to
   its maximum value, and has its minimum value whenever the information
   being modeled is smaller than or equal to its minimum value.  If the
   information being modeled subsequently decreases below (increases
   above) the maximum (minimum) value, the Gauge32 also decreases
   (increases).  (Note that despite of the use of the term "latched" in
   the original definition of this type, it does not become "stuck" at
   its maximum or minimum value.)

As you can read 4294967295 is 2^32-1 and a valid value , you should also note that Gauge32 represents a non-negative integer . ( 32 bits) so it seems like it can never be -1

regarding this information snmpcollector seems to be returning the correct value.

matejv commented 2 years ago

Thank you @toni-moreno

I understand this. The problem is that the SNMP device I'm polling is returning data type wrong. It should not be Gauge32, but Integer32 (signed, to allow for negative values).

I'm unlikely to be able to get the vendor to fix their device unfortunately. So I thought SNMPcollector would be able to:

I was under the impression that setting DataSrcType to Integer32 would achieve this.

image

If may I ask - what do the different (SNMP SMI Type) *Integer* options actually do? If the value data type is determined by the snmp type sent by device. It doesn't seem to make any difference of I choose here Integer32 or UInteger32, if device sends Gauge32. Correct?

toni-moreno commented 2 years ago

Hello again @matejv , sorry for the late response.

Unfortunately for you snmpcollector doesn't force sign or size conversion , it handle data with its original type always .

you can see code yourself.

https://github.com/toni-moreno/snmpcollector/blob/c49cfc1858f56591bc5adf296151198b56355947/pkg/data/metric/snmpmetric.go#L359-L372

and here how to decode data.

https://github.com/toni-moreno/snmpcollector/blob/c49cfc1858f56591bc5adf296151198b56355947/pkg/data/snmp/snmp.go#L436-L471

and not sure if this policy should change in the future, because this would be a big breaking change.

instead you could create a new STRING EVAL field to force yourself conversion from the original , by only creating a evaluation expression like my_orig_field > 4294967295 ? ( 4294967295 - my_orig_field )

https://github.com/toni-moreno/snmpcollector/wiki/Component:-SNMP-Metrics#about-stringeval-metrics

Of course you don't need to send my_orig_field to the backend DB , you can choose not to send in the measurement configuration , choosing "never report" my_orig_field. image

Let me know if this STRING EVAL new field would resolve your issue.

matejv commented 2 years ago

I was looking at the code and I suspected as much, but I'm not really used to golang so I wanted to make sure.

I agree that changing current behavior might break stuff for many users, so best not to change it. In the end the fault lies in the buggy device I am using.

I did end up using your STRINGEVAL suggestion. I've done some maths and came up with a bit different formula:

(my_orig_field > 2147483647) ? (my_orig_field - 4294967294) : my_orig_field

2147483647 is the largest possible 32bit signed int. If we know we are operating with signed numbers, then any value larger than that must be negative. I've done some tests and checked results via this form also. If anyone else in the future will stumble across this issue.

Thany ou very much for your help and for creating this amazing tool. Best SNMP poller I've ever used!