search

toni-moreno / snmpcollector

A full featured Generic SNMP data collector with Web Administration Interface for InfluxDB
MIT License
288 stars 53 forks source link

Error on getting initial basic system, write: invalid argument #395

Closed adrianalcalde closed 3 years ago

adrianalcalde commented 5 years ago

I'm using version 0.8.0, a little more than 2000 devices. Seeing this error for some devices. Do you think it can have something to do with the amount of devices I am polling? I can have a distributed setup and break apart the collectors. Currently, I have a single influx measurement with the following metric fields, at 5 minute interval:

Example log of one device with the issue: time="2019-03-10 10:25:00" level=info msg="SNMPDEVICE [] Beginning SNMP connection for measurement init" time="2019-03-10 10:25:00" level=info msg="First SNMP connection to host 10.24.1.207 stablished with MaxRepetitions set to 50" time="2019-03-10 10:25:00" level=error msg="Error on getting initial basic system, Info to device : write udp 10.38.98.45:37965->10.24.1.207:161: write: invalid argument" time="2019-03-10 10:25:00" level=error msg="error on get System Info write udp 10.38.98.45:37965->10.24.1.207:161: write: invalid argument" time="2019-03-10 10:25:00" level=error msg="SNMPDEVICE [] Client connect error to device error :write udp 10.38.98.45:37965->10.24.1.207:161: write: invalid argument"

snmpwalk of device in cli responds correctly.

I'm also seeing another issue where if more than 1 measurement filter is applied to a snmpdevice, only one of the filters is processed. I'm not sure if its happening to anyone else with the 0.8.0 version or just unique to my setup.

If you need any other info, let me know.

Thanks

adrianalcalde commented 5 years ago

@toni-moreno @sbengo Hey guys, any ideas? I can provide any other details you need from our setup. Not happening to a lot of devices, but happening on a few constantly. Total current devices are a little over 1000. About 50 devices where this is happening. Below is a debug log sample of what happens on the poll:

time="2019-03-18 12:55:00" level=debug msg="MEASUREMENT [interfaces_snmp_influx] generating influx point for indexed 1612202008" time="2019-03-18 12:55:00" level=debug msg="MEASUREMENT [interfaces_snmp_influx] IDX :&{Valid:false Data:map[ifOperStatus:0xc0310fe2d0 ifName:0xc02f654ff0 ifAlias:0xc02f6553b0 ifType:0xc0310fe000 ifPhysAddress:0xc0310fe0f0 ifHCOutOctets:0xc02f6551d0 ifDescr:0xc0068f83c0 ifIndex:0xc0068f8ff0 ifAdminStatus:0xc0310fe1e0 ifOutErrors:0xc02f654f00 ifHighSpeed:0xc02f6552c0 tmnxPortLagId:0xc00f167e00 ifInErrors:0xc02f654e10 ifHCInOctets:0xc02f6550e0 tmnxPortMode:0xc0068f9590]}" time="2019-03-18 12:55:00" level=debug msg="DEBUG METRIC CONFIG &{ID:ifInErrors FieldName:ifInErrors Description: BaseOID:.1.3.6.1.2.1.2.2.1.14 DataSrcType:COUNTERXX GetRate:true Scale:0 Shift:0 IsTag:false ExtraData: Conversion:0 Names:map[]}" time="2019-03-18 12:55:00" level=warning msg="Warning METRIC ID [ifInErrors] from MEASUREMENT[ interfaces_snmp_influx ] with TAGS [map[device:M1000-MIAVFLFF-A7950A system_address:208.67.164.131 mgmt_ip:10.24.1.195 ifIndex:1612202008]] has obsolete data => See Metric Runtime [ &{cfg:0xc0002642d0 Valid:false CookedValue:0 CurValue:32 LastValue:32 CurTime:2019-03-18 12:50:00.001034366 -0400 EDT m=+355145.043981791 LastTime:2019-03-18 12:46:03.490669596 -0400 EDT m=+354908.533617015 ElapsedTime:236.510364776 Compute:0x946940 Scale:0x9452f0 Convert:0x948af0 SetRawData:0x9465d0 RealOID:.1.3.6.1.2.1.2.2.1.14.1612202008 Report:1 re: mm:[] expr: condflt: log:0xc02396b270} ]" time="2019-03-18 12:55:00" level=debug msg="DEBUG METRIC CONFIG &{ID:ifHCInOctets FieldName:ifHCInOctets Description: BaseOID:.1.3.6.1.2.1.31.1.1.1.6 DataSrcType:COUNTERXX GetRate:true Scale:0 Shift:0 IsTag:false ExtraData: Conversion:0 Names:map[]}" time="2019-03-18 12:55:00" level=warning msg="Warning METRIC ID [ifHCInOctets] from MEASUREMENT[ interfaces_snmp_influx ] with TAGS [map[device:M1000-MIAVFLFF-A7950A system_address:208.67.164.131 mgmt_ip:10.24.1.195 ifIndex:1612202008]] has obsolete data => See Metric Runtime [ &{cfg:0xc000264120 Valid:false CookedValue:4.688524705292428e+08 CurValue:2192495630290542 LastValue:2192384741821711 CurTime:2019-03-18 12:50:00.001034366 -0400 EDT m=+355145.043981791 LastTime:2019-03-18 12:46:03.490669596 -0400 EDT m=+354908.533617015 ElapsedTime:236.510364776 Compute:0x946940 Scale:0x9452f0 Convert:0x948af0 SetRawData:0x9465d0 RealOID:.1.3.6.1.2.1.31.1.1.1.6.1612202008 Report:1 re: mm:[] expr: condflt: log:0xc02396b270} ]" time="2019-03-18 12:55:00" level=debug msg="DEBUG METRIC CONFIG &{ID:tmnxPortMode FieldName:tmnxPortMode Description: BaseOID:.1.3.6.1.4.1.6527.3.1.2.2.4.2.1.11.1 DataSrcType:INTEGER GetRate:false Scale:0 Shift:0 IsTag:true ExtraData: Conversion:0 Names:map[]}" time="2019-03-18 12:55:00" level=warning msg="Warning METRIC ID [tmnxPortMode] from MEASUREMENT[ interfaces_snmp_influx ] with TAGS [map[ifIndex:1612202008 device:M1000-MIAVFLFF-A7950A system_address:208.67.164.131 mgmt_ip:10.24.1.195]] has obsolete data => See Metric Runtime [ &{cfg:0xc00022fd40 Valid:false CookedValue:2 CurValue: LastValue: CurTime:2019-03-18 12:50:00.001034366 -0400 EDT m=+355145.043981791 LastTime:0001-01-01 00:00:00 +0000 UTC ElapsedTime:0 Compute: Scale:0x9452f0 Convert:0x948ab0 SetRawData:0x945570 RealOID:.1.3.6.1.4.1.6527.3.1.2.2.4.2.1.11.1.1612202008 Report:1 re: mm:[] expr: condflt: log:0xc02396b270} ]" time="2019-03-18 12:55:00" level=debug msg="DEBUG METRIC CONFIG &{ID:ifOperStatus FieldName:ifOperStatus Description: BaseOID:.1.3.6.1.2.1.2.2.1.8 DataSrcType:Integer32 GetRate:false Scale:0 Shift:0 IsTag:false ExtraData: Conversion:1 Names:map[]}" time="2019-03-18 12:55:00" level=warning msg="Warning METRIC ID [ifOperStatus] from MEASUREMENT[ interfaces_snmp_influx ] with TAGS [map[device:M1000-MIAVFLFF-A7950A system_address:208.67.164.131 mgmt_ip:10.24.1.195 ifIndex:1612202008]] has obsolete data => See Metric Runtime [ &{cfg:0xc000264360 Valid:false CookedValue:1 CurValue: LastValue: CurTime:2019-03-18 12:50:00.001034366 -0400 EDT m=+355145.043981791 LastTime:0001-01-01 00:00:00 +0000 UTC ElapsedTime:0 Compute: Scale:0x9452f0 Convert:0x948ab0 SetRawData:0x945570 RealOID:.1.3.6.1.2.1.2.2.1.8.1612202008 Report:1 re: mm:[] expr: condflt: log:0xc02396b270} ]" time="2019-03-18 12:55:00" level=debug msg="DEBUG METRIC CONFIG &{ID:ifName FieldName:ifName Description: BaseOID:.1.3.6.1.2.1.31.1.1.1.1 DataSrcType:OCTETSTRING GetRate:false Scale:0 Shift:0 IsTag:true ExtraData: Conversion:3 Names:map[]}" time="2019-03-18 12:55:00" level=warning msg="Warning METRIC ID [ifName] from MEASUREMENT[ interfaces_snmp_influx ] with TAGS [map[mgmt_ip:10.24.1.195 ifIndex:1612202008 device:M1000-MIAVFLFF-A7950A system_address:208.67.164.131]] has obsolete data => See Metric Runtime [ &{cfg:0xc0002641b0 Valid:false CookedValue:6/1/3 CurValue: LastValue: CurTime:2019-03-18 12:50:00.001034366 -0400 EDT m=+355145.043981791 LastTime:0001-01-01 00:00:00 +0000 UTC ElapsedTime:0 Compute: Scale:0x9452f0 Convert:0x948ab0 SetRawData:0x9477f0 RealOID:.1.3.6.1.2.1.31.1.1.1.1.1612202008 Report:1 re: mm:[] expr: condflt: log:0xc02396b270} ]" time="2019-03-18 12:55:00" level=debug msg="DEBUG METRIC CONFIG &{ID:ifAlias FieldName:ifAlias Description: BaseOID:.1.3.6.1.2.1.31.1.1.1.18 DataSrcType:OCTETSTRING GetRate:false Scale:0 Shift:0 IsTag:false ExtraData: Conversion:3 Names:map[]}" time="2019-03-18 12:55:00" level=warning msg="Warning METRIC ID [ifAlias] from MEASUREMENT[ interfaces_snmp_influx ] with TAGS [map[device:M1000-MIAVFLFF-A7950A system_address:208.67.164.131 mgmt_ip:10.24.1.195 ifIndex:1612202008]] has obsolete data => See Metric Runtime [ &{cfg:0xc00022fef0 Valid:false CookedValue:M1000-MTS to M1049B-CSEA 1/1/1 CurValue: LastValue: CurTime:2019-03-18 12:50:00.001034366 -0400 EDT m=+355145.043981791 LastTime:0001-01-01 00:00:00 +0000 UTC ElapsedTime:0 Compute: Scale:0x9452f0 Convert:0x948ab0 SetRawData:0x9477f0 RealOID:.1.3.6.1.2.1.31.1.1.1.18.1612202008 Report:1 re: mm:[] expr: condflt: log:0xc02396b270} ]" time="2019-03-18 12:55:00" level=debug msg="DEBUG METRIC CONFIG &{ID:ifDescr FieldName:ifDescr Description: BaseOID:.1.3.6.1.2.1.2.2.1.2 DataSrcType:OCTETSTRING GetRate:false Scale:0 Shift:0 IsTag:false ExtraData: Conversion:3 Names:map[]}" time="2019-03-18 12:55:00" level=warning msg="Warning METRIC ID [ifDescr] from MEASUREMENT[ interfaces_snmp_influx ] with TAGS [map[ifIndex:1612202008 device:M1000-MIAVFLFF-A7950A system_address:208.67.164.131 mgmt_ip:10.24.1.195]] has obsolete data => See Metric Runtime [ &{cfg:0xc00022fe60 Valid:false CookedValue:6/1/3, 100-Gig Ethernet, "M1000-MTS to M1049B-CSEA 1/1/1" CurValue: LastValue: CurTime:2019-03-18 12:50:00.001034366 -0400 EDT m=+355145.043981791 LastTime:0001-01-01 00:00:00 +0000 UTC ElapsedTime:0 Compute: Scale:0x9452f0 Convert:0x948ab0 SetRawData:0x9477f0 RealOID:.1.3.6.1.2.1.2.2.1.2.1612202008 Report:1 re: mm:[] expr: condflt: log:0xc02396b270} ]" time="2019-03-18 12:55:00" level=debug msg="DEBUG METRIC CONFIG &{ID:ifIndex FieldName:ifIndex Description: BaseOID:.1.3.6.1.2.1.2.2.1.1 DataSrcType:Integer32 GetRate:false Scale:0 Shift:0 IsTag:true ExtraData: Conversion:0 Names:map[]}" time="2019-03-18 12:55:00" level=warning msg="Warning METRIC ID [ifIndex] from MEASUREMENT[ interfaces_snmp_influx ] with TAGS [map[device:M1000-MIAVFLFF-A7950A system_address:208.67.164.131 mgmt_ip:10.24.1.195 ifIndex:1612202008]] has obsolete data => See Metric Runtime [ &{cfg:0xc00022fdd0 Valid:false CookedValue:1.612202008e+09 CurValue: LastValue: CurTime:2019-03-18 12:50:00.001034366 -0400 EDT m=+355145.043981791 LastTime:0001-01-01 00:00:00 +0000 UTC ElapsedTime:0 Compute: Scale:0x9452f0 Convert:0x948ab0 SetRawData:0x945570 RealOID:.1.3.6.1.2.1.2.2.1.1.1612202008 Report:1 re: mm:[] expr: condflt: log:0xc02396b270} ]" time="2019-03-18 12:55:00" level=debug msg="DEBUG METRIC CONFIG &{ID:ifType FieldName:ifType Description: BaseOID:.1.3.6.1.2.1.2.2.1.3 DataSrcType:Integer32 GetRate:false Scale:0 Shift:0 IsTag:true ExtraData: Conversion:0 Names:map[]}" time="2019-03-18 12:55:00" level=warning msg="Warning METRIC ID [ifType] from MEASUREMENT[ interfaces_snmp_influx ] with TAGS [map[ifIndex:1612202008 device:M1000-MIAVFLFF-A7950A system_address:208.67.164.131 mgmt_ip:10.24.1.195]] has obsolete data => See Metric Runtime [ &{cfg:0xc000264510 Valid:false CookedValue:6 CurValue: LastValue: CurTime:2019-03-18 12:50:00.001034366 -0400 EDT m=+355145.043981791 LastTime:0001-01-01 00:00:00 +0000 UTC ElapsedTime:0 Compute: Scale:0x9452f0 Convert:0x948ab0 SetRawData:0x945570 RealOID:.1.3.6.1.2.1.2.2.1.3.1612202008 Report:1 re: mm:[] expr: condflt: log:0xc02396b270} ]" time="2019-03-18 12:55:00" level=debug msg="DEBUG METRIC CONFIG &{ID:ifPhysAddress FieldName:ifPhysAddress Description: BaseOID:.1.3.6.1.2.1.2.2.1.6 DataSrcType:HWADDR GetRate:false Scale:0 Shift:0 IsTag:false ExtraData: Conversion:3 Names:map[]}" time="2019-03-18 12:55:00" level=warning msg="Warning METRIC ID [ifPhysAddress] from MEASUREMENT[ interfaces_snmp_influx ] with TAGS [map[device:M1000-MIAVFLFF-A7950A system_address:208.67.164.131 mgmt_ip:10.24.1.195 ifIndex:1612202008]] has obsolete data => See Metric Runtime [ &{cfg:0xc000264480 Valid:false CookedValue:a4:7b:2c:c4:47:e1 CurValue: LastValue: CurTime:2019-03-18 12:50:00.001034366 -0400 EDT m=+355145.043981791 LastTime:0001-01-01 00:00:00 +0000 UTC ElapsedTime:0 Compute: Scale:0x9452f0 Convert:0x948ab0 SetRawData:0x947db0 RealOID:.1.3.6.1.2.1.2.2.1.6.1612202008 Report:1 re: mm:[] expr: condflt: log:0xc02396b270} ]" time="2019-03-18 12:55:00" level=debug msg="DEBUG METRIC CONFIG &{ID:ifHCOutOctets FieldName:ifHCOutOctets Description: BaseOID:.1.3.6.1.2.1.31.1.1.1.10 DataSrcType:COUNTERXX GetRate:true Scale:0 Shift:0 IsTag:false ExtraData: Conversion:0 Names:map[]}" time="2019-03-18 12:55:00" level=warning msg="Warning METRIC ID [ifHCOutOctets] from MEASUREMENT[ interfaces_snmp_influx ] with TAGS [map[system_address:208.67.164.131 mgmt_ip:10.24.1.195 ifIndex:1612202008 device:M1000-MIAVFLFF-A7950A]] has obsolete data => See Metric Runtime [ &{cfg:0xc000264090 Valid:false CookedValue:1.1958133757726102e+09 CurValue:5581190726260001 LastValue:5580907904002293 CurTime:2019-03-18 12:50:00.001034366 -0400 EDT m=+355145.043981791 LastTime:2019-03-18 12:46:03.490669596 -0400 EDT m=+354908.533617015 ElapsedTime:236.510364776 Compute:0x946940 Scale:0x9452f0 Convert:0x948af0 SetRawData:0x9465d0 RealOID:.1.3.6.1.2.1.31.1.1.1.10.1612202008 Report:1 re: mm:[] expr: condflt: log:0xc02396b270} ]" time="2019-03-18 12:55:00" level=debug msg="DEBUG METRIC CONFIG &{ID:tmnxPortLagId FieldName:tmnxPortLagId Description: BaseOID:.1.3.6.1.4.1.6527.3.1.2.2.4.2.1.13.1 DataSrcType:Unsigned32 GetRate:false Scale:0 Shift:0 IsTag:true ExtraData: Conversion:0 Names:map[]}" time="2019-03-18 12:55:00" level=warning msg="Warning METRIC ID [tmnxPortLagId] from MEASUREMENT[ interfaces_snmp_influx ] with TAGS [map[device:M1000-MIAVFLFF-A7950A system_address:208.67.164.131 mgmt_ip:10.24.1.195 ifIndex:1612202008]] has obsolete data => See Metric Runtime [ &{cfg:0xc00022fcb0 Valid:false CookedValue:0 CurValue: LastValue: CurTime:2019-03-18 12:50:00.001034366 -0400 EDT m=+355145.043981791 LastTime:0001-01-01 00:00:00 +0000 UTC ElapsedTime:0 Compute: Scale:0x9452f0 Convert:0x948ab0 SetRawData:0x945690 RealOID:.1.3.6.1.4.1.6527.3.1.2.2.4.2.1.13.1.1612202008 Report:1 re: mm:[] expr: condflt: log:0xc02396b270} ]" time="2019-03-18 12:55:00" level=debug msg="DEBUG METRIC CONFIG &{ID:ifAdminStatus FieldName:ifAdminStatus Description: BaseOID:.1.3.6.1.2.1.2.2.1.7 DataSrcType:Integer32 GetRate:false Scale:0 Shift:0 IsTag:false ExtraData: Conversion:1 Names:map[]}" time="2019-03-18 12:55:00" level=warning msg="Warning METRIC ID [ifAdminStatus] from MEASUREMENT[ interfaces_snmp_influx ] with TAGS [map[device:M1000-MIAVFLFF-A7950A system_address:208.67.164.131 mgmt_ip:10.24.1.195 ifIndex:1612202008]] has obsolete data => See Metric Runtime [ &{cfg:0xc0002643f0 Valid:false CookedValue:1 CurValue: LastValue: CurTime:2019-03-18 12:50:00.001034366 -0400 EDT m=+355145.043981791 LastTime:0001-01-01 00:00:00 +0000 UTC ElapsedTime:0 Compute: Scale:0x9452f0 Convert:0x948ab0 SetRawData:0x945570 RealOID:.1.3.6.1.2.1.2.2.1.7.1612202008 Report:1 re: mm:[] expr: condflt: log:0xc02396b270} ]" time="2019-03-18 12:55:00" level=debug msg="DEBUG METRIC CONFIG &{ID:ifOutErrors FieldName:ifOutErrors Description: BaseOID:.1.3.6.1.2.1.2.2.1.20 DataSrcType:COUNTERXX GetRate:true Scale:0 Shift:0 IsTag:false ExtraData: Conversion:0 Names:map[]}" time="2019-03-18 12:55:00" level=warning msg="Warning METRIC ID [ifOutErrors] from MEASUREMENT[ interfaces_snmp_influx ] with TAGS [map[device:M1000-MIAVFLFF-A7950A system_address:208.67.164.131 mgmt_ip:10.24.1.195 ifIndex:1612202008]] has obsolete data => See Metric Runtime [ &{cfg:0xc000264240 Valid:false CookedValue:0 CurValue:0 LastValue:0 CurTime:2019-03-18 12:50:00.001034366 -0400 EDT m=+355145.043981791 LastTime:2019-03-18 12:46:03.490669596 -0400 EDT m=+354908.533617015 ElapsedTime:236.510364776 Compute:0x946940 Scale:0x9452f0 Convert:0x948af0 SetRawData:0x9465d0 RealOID:.1.3.6.1.2.1.2.2.1.20.1612202008 Report:1 re: mm:[] expr: condflt: log:0xc02396b270} ]" time="2019-03-18 12:55:00" level=debug msg="DEBUG METRIC CONFIG &{ID:ifHighSpeed FieldName:ifHighSpeed Description: BaseOID:.1.3.6.1.2.1.31.1.1.1.15 DataSrcType:Gauge32 GetRate:false Scale:0 Shift:0 IsTag:false ExtraData: Conversion:0 Names:map[]}" time="2019-03-18 12:55:00" level=warning msg="Warning METRIC ID [ifHighSpeed] from MEASUREMENT[ interfaces_snmp_influx ] with TAGS [map[device:M1000-MIAVFLFF-A7950A system_address:208.67.164.131 mgmt_ip:10.24.1.195 ifIndex:1612202008]] has obsolete data => See Metric Runtime [ &{cfg:0xc000264000 Valid:false CookedValue:100000 CurValue: LastValue: CurTime:2019-03-18 12:50:00.001034366 -0400 EDT m=+355145.043981791 LastTime:0001-01-01 00:00:00 +0000 UTC ElapsedTime:0 Compute: Scale:0x9452f0 Convert:0x948ab0 SetRawData:0x945690 RealOID:.1.3.6.1.2.1.31.1.1.1.15.1612202008 Report:1 re: mm:[] expr: condflt: log:0xc02396b270} ]" time="2019-03-18 12:55:00" level=debug msg="MEASUREMENT [interfaces_snmp_influx] FIELDS:map[] TAGS:map[device:M1000-MIAVFLFF-A7950A system_address:208.67.164.131 mgmt_ip:10.24.1.195 ifIndex:1612202008]" time="2019-03-18 12:55:00" level=warning msg="MEASUREMENT [interfaces_snmp_influx] error in influx point creation :point without fields is unsupported"

toni-moreno commented 5 years ago

Hi @adrianalcalde sorry by the late response. We are right now really busy , but if you can update the issue I will try to review in a few days.

Thank you very much

toni-moreno commented 3 years ago

Hello @adrianalcalde, I will close this issue, please update to the last version and reopen it if issue persist