Closed hyberdk closed 5 years ago
Hi @hyberdk ,
As you say, its quite strange and I'm not sure if its related with the SNMPCollector.
Before entering into metric config:
influx -username '<USER>' -password '<PASSWORD>' -format 'csv' -database <DB> -execute 'SELECT* FROM "<DB>"."<RP>".ip_sla_core_switches where time > now() - 5m;
Thanks, regards!
Hi @sbengo ,
thanks for your reply, my data looks like this:
root@django-dev:/home/test# docker exec -it influxdb influx -format 'csv' -database snmpcollector -execute 'select * from snmpcollector.autogen.ip_sla_core_switches where time > now() - 5m;'
name,time,hostname,rttMonCtrlAdminTag,rttMonLatestRttOperAddress,rttMonLatestRttOperApplSpecificSense,rttMonLatestRttOperCompletionTime,rttMonLatestRttOperSense,rttMonLatestRttOperTime
",0,250,1,172264104s,1555182180000000000,XYZ-DMVPN01,AAA File server,"
",0,58,1,172264086es,1555182180000000000,XYZ-DMVPN01,BBB File server,"
",0,59,1,172264087es,1555182180000000000,XYZ-DMVPN01,BBB Outlook,"
",0,59,1,172264086es,1555182180000000000,XYZ-DMVPN01,BBB SAP,"
",0,62,1,172264087es,1555182180000000000,XYZ-DMVPN01,BBB SCCM/Backup ,"
",0,60,1,172264086es,1555182180000000000,XYZ-DMVPN01,BBB Tandberg Video,"
",0,44,1,172294086es,1555182180000000000,XYZ-DMVPN01,BBB Voice,"
",0,247,1,172264106s,1555182180000000000,XYZ-DMVPN01,CCC File server,"
",0,0,4,172294080hes,1555182180000000000,XYZ-DMVPN01,THEHUB,"
",0,0,4,172294080hes,1555182180000000000,XYZ-DMVPN01,WWW.CISCO.COM,"
",0,250,1,172264104s,1555182240000000000,XYZ-DMVPN01,AAA File server,"
",0,58,1,172264086es,1555182240000000000,XYZ-DMVPN01,BBB File server,"
",0,59,1,172264087es,1555182240000000000,XYZ-DMVPN01,BBB Outlook,"
",0,59,1,172264086es,1555182240000000000,XYZ-DMVPN01,BBB SAP,"
",0,62,1,172264087es,1555182240000000000,XYZ-DMVPN01,BBB SCCM/Backup ,"
",0,60,1,172264086es,1555182240000000000,XYZ-DMVPN01,BBB Tandberg Video,"
",0,44,1,172294086es,1555182240000000000,XYZ-DMVPN01,BBB Voice,"
",0,247,1,172264106s,1555182240000000000,XYZ-DMVPN01,CCC File server,"
",0,0,4,172294080hes,1555182240000000000,XYZ-DMVPN01,THEHUB,"
",0,0,4,172294080hes,1555182240000000000,XYZ-DMVPN01,WWW.CISCO.COM,"
",0,250,1,172264104s,1555182300000000000,XYZ-DMVPN01,AAA File server,"
",0,58,1,172264086es,1555182300000000000,XYZ-DMVPN01,BBB File server,"
",0,59,1,172264087es,1555182300000000000,XYZ-DMVPN01,BBB Outlook,"
",0,59,1,172264086es,1555182300000000000,XYZ-DMVPN01,BBB SAP,"
",0,62,1,172264087es,1555182300000000000,XYZ-DMVPN01,BBB SCCM/Backup ,"
",0,60,1,172264086es,1555182300000000000,XYZ-DMVPN01,BBB Tandberg Video,"
",0,44,1,172294086es,1555182300000000000,XYZ-DMVPN01,BBB Voice,"
",0,247,1,172264106s,1555182300000000000,XYZ-DMVPN01,CCC File server,"
",0,0,4,172294080hes,1555182300000000000,XYZ-DMVPN01,THEHUB,"
",0,0,4,172294080hes,1555182300000000000,XYZ-DMVPN01,WWW.CISCO.COM,"
",0,250,1,172264104s,1555182360000000000,XYZ-DMVPN01,AAA File server,"
",0,58,1,172264086es,1555182360000000000,XYZ-DMVPN01,BBB File server,"
",0,59,1,172264087es,1555182360000000000,XYZ-DMVPN01,BBB Outlook,"
",0,59,1,172264086es,1555182360000000000,XYZ-DMVPN01,BBB SAP,"
",0,62,1,172264087es,1555182360000000000,XYZ-DMVPN01,BBB SCCM/Backup ,"
",0,60,1,172264086es,1555182360000000000,XYZ-DMVPN01,BBB Tandberg Video,"
",0,44,1,172294086es,1555182360000000000,XYZ-DMVPN01,BBB Voice,"
",0,247,1,172264106s,1555182360000000000,XYZ-DMVPN01,CCC File server,"
",0,0,4,172294080hes,1555182360000000000,XYZ-DMVPN01,THEHUB,"
",0,0,4,172294080hes,1555182360000000000,XYZ-DMVPN01,WWW.CISCO.COM,"
",0,238,1,172294104s,1555182420000000000,XYZ-DMVPN01,AAA File server,"
",0,62,1,172294086es,1555182420000000000,XYZ-DMVPN01,BBB File server,"
",0,62,1,172294086es,1555182420000000000,XYZ-DMVPN01,BBB Outlook,"
",0,55,1,172294086es,1555182420000000000,XYZ-DMVPN01,BBB SAP,"
",0,66,1,172294086es,1555182420000000000,XYZ-DMVPN01,BBB SCCM/Backup ,"
",0,61,1,172294086es,1555182420000000000,XYZ-DMVPN01,BBB Tandberg Video,"
",0,44,1,172294086es,1555182420000000000,XYZ-DMVPN01,BBB Voice,"
",0,247,1,172264106s,1555182420000000000,XYZ-DMVPN01,CCC File server,"
",0,0,4,172294080hes,1555182420000000000,XYZ-DMVPN01,THEHUB,"
",0,0,4,172294080hes,1555182420000000000,XYZ-DMVPN01,WWW.CISCO.COM,"
root@django-dev:/home/test#
Im no Influx expert (my far), but does the order not look strange?
Actually I tried to replicate this on another server, but there it looks fine..
root@test-linux1:/home/test# docker exec -it influxdb influx -format 'csv' -database snmpcollector -execute 'select * from snmpcollector_test.autogen.ipsla_core where time > now() - 5m;'
name,time,hostname,rttMonCtrlAdminTag,rttMonLatestRttOperApplSpecificSense,rttMonLatestRttOperCompletionTime,rttMonLatestRttOperSense,rttMonLatestRttOperSenseDescription,rttMonLatestRttOperTime
ipsla_core,1555182780000000000,XYZ-DMVPN01,AAA File server,0,246,ok(1),,172324101
ipsla_core,1555182780000000000,XYZ-DMVPN01,BBB File server,0,59,ok(1),,172324084
ipsla_core,1555182780000000000,XYZ-DMVPN01,BBB Outlook,0,56,ok(1),,172324087
ipsla_core,1555182780000000000,XYZ-DMVPN01,BBB SAP,0,65,ok(1),,172324084
ipsla_core,1555182780000000000,XYZ-DMVPN01,BBB SCCM/Backup ,0,54,ok(1),,172324087
ipsla_core,1555182780000000000,XYZ-DMVPN01,BBB Tandberg Video,0,65,ok(1),,172324084
ipsla_core,1555182780000000000,XYZ-DMVPN01,BBB Voice,0,45,ok(1),,172354084
ipsla_core,1555182780000000000,XYZ-DMVPN01,CCC File server,0,255,ok(1),,172324104
ipsla_core,1555182780000000000,XYZ-DMVPN01,THEHUB,0,0,timeout(4),,172354080
ipsla_core,1555182780000000000,XYZ-DMVPN01,WWW.CISCO.COM,0,0,timeout(4),,172354080
ipsla_core,1555182840000000000,XYZ-DMVPN01,AAA File server,0,246,ok(1),,172324101
ipsla_core,1555182840000000000,XYZ-DMVPN01,BBB File server,0,59,ok(1),,172324084
ipsla_core,1555182840000000000,XYZ-DMVPN01,BBB Outlook,0,56,ok(1),,172324087
ipsla_core,1555182840000000000,XYZ-DMVPN01,BBB SAP,0,65,ok(1),,172324084
ipsla_core,1555182840000000000,XYZ-DMVPN01,BBB SCCM/Backup ,0,54,ok(1),,172324087
ipsla_core,1555182840000000000,XYZ-DMVPN01,BBB Tandberg Video,0,65,ok(1),,172324084
ipsla_core,1555182840000000000,XYZ-DMVPN01,BBB Voice,0,45,ok(1),,172354084
ipsla_core,1555182840000000000,XYZ-DMVPN01,CCC File server,0,255,ok(1),,172324104
ipsla_core,1555182840000000000,XYZ-DMVPN01,THEHUB,0,0,timeout(4),,172354080
ipsla_core,1555182840000000000,XYZ-DMVPN01,WWW.CISCO.COM,0,0,timeout(4),,172354080
ipsla_core,1555182900000000000,XYZ-DMVPN01,AAA File server,0,246,ok(1),,172324101
ipsla_core,1555182900000000000,XYZ-DMVPN01,BBB File server,0,59,ok(1),,172324084
ipsla_core,1555182900000000000,XYZ-DMVPN01,BBB Outlook,0,56,ok(1),,172324087
ipsla_core,1555182900000000000,XYZ-DMVPN01,BBB SAP,0,65,ok(1),,172324084
ipsla_core,1555182900000000000,XYZ-DMVPN01,BBB SCCM/Backup ,0,54,ok(1),,172324087
ipsla_core,1555182900000000000,XYZ-DMVPN01,BBB Tandberg Video,0,65,ok(1),,172324084
ipsla_core,1555182900000000000,XYZ-DMVPN01,BBB Voice,0,45,ok(1),,172354084
ipsla_core,1555182900000000000,XYZ-DMVPN01,CCC File server,0,255,ok(1),,172324104
ipsla_core,1555182900000000000,XYZ-DMVPN01,THEHUB,0,0,timeout(4),,172354080
ipsla_core,1555182900000000000,XYZ-DMVPN01,WWW.CISCO.COM,0,0,timeout(4),,172354080
ipsla_core,1555182960000000000,XYZ-DMVPN01,AAA File server,0,246,ok(1),,172324101
ipsla_core,1555182960000000000,XYZ-DMVPN01,BBB File server,0,59,ok(1),,172324084
ipsla_core,1555182960000000000,XYZ-DMVPN01,BBB Outlook,0,56,ok(1),,172324087
ipsla_core,1555182960000000000,XYZ-DMVPN01,BBB SAP,0,65,ok(1),,172324084
ipsla_core,1555182960000000000,XYZ-DMVPN01,BBB SCCM/Backup ,0,54,ok(1),,172324087
ipsla_core,1555182960000000000,XYZ-DMVPN01,BBB Tandberg Video,0,65,ok(1),,172324084
ipsla_core,1555182960000000000,XYZ-DMVPN01,BBB Voice,0,45,ok(1),,172354084
ipsla_core,1555182960000000000,XYZ-DMVPN01,CCC File server,0,255,ok(1),,172324104
ipsla_core,1555182960000000000,XYZ-DMVPN01,THEHUB,0,0,timeout(4),,172354080
ipsla_core,1555182960000000000,XYZ-DMVPN01,WWW.CISCO.COM,0,0,timeout(4),,172354080
ipsla_core,1555183020000000000,XYZ-DMVPN01,AAA File server,0,246,ok(1),,172324101
ipsla_core,1555183020000000000,XYZ-DMVPN01,BBB File server,0,57,ok(1),,172354084
ipsla_core,1555183020000000000,XYZ-DMVPN01,BBB Outlook,0,60,ok(1),,172354085
ipsla_core,1555183020000000000,XYZ-DMVPN01,BBB SAP,0,60,ok(1),,172354084
ipsla_core,1555183020000000000,XYZ-DMVPN01,BBB SCCM/Backup ,0,59,ok(1),,172354085
ipsla_core,1555183020000000000,XYZ-DMVPN01,BBB Tandberg Video,0,62,ok(1),,172354084
ipsla_core,1555183020000000000,XYZ-DMVPN01,BBB Voice,0,45,ok(1),,172354084
ipsla_core,1555183020000000000,XYZ-DMVPN01,CCC File server,0,253,ok(1),,172354105
ipsla_core,1555183020000000000,XYZ-DMVPN01,THEHUB,0,0,timeout(4),,172354080
ipsla_core,1555183020000000000,XYZ-DMVPN01,WWW.CISCO.COM,0,0,timeout(4),,172354080
root@test-linux1:/home/test#
The system that has the issue is polling ~2800 devices, my test system that does not have the issue as well 1 ;-)
Esben
Hi @hyberdk , thanks or the report!
I'm not sure what is happening but it seems not related with SNMPCollector itself...
Thanks, Regards!
Hi @sbengo ,
here is the data collected from another host
# influx -host django-dev -format 'csv' -database snmpcollector -execute 'select * from snmpcollector.autogen.ip_sla_core_switches where time > now() - 5m;'
name,time,hostname,rttMonCtrlAdminTag,rttMonLatestRttOperAddress,rttMonLatestRttOperApplSpecificSense,rttMonLatestRttOperCompletionTime,rttMonLatestRttOperSense,rttMonLatestRttOperTime
",0,241,1,202744102s,1555487160000000000,XYZ-DMVPN01,AAA File server,"
",0,56,1,202744085es,1555487160000000000,XYZ-DMVPN01,BBB File server,"
",0,58,1,202744085es,1555487160000000000,XYZ-DMVPN01,BBB Outlook,"
",0,54,1,202744085es,1555487160000000000,XYZ-DMVPN01,BBB SAP,"
",0,60,1,202744086es,1555487160000000000,XYZ-DMVPN01,BBB SCCM/Backup ,"
",0,58,1,202744086es,1555487160000000000,XYZ-DMVPN01,BBB Tandberg Video,"
",0,46,1,202774084es,1555487160000000000,XYZ-DMVPN01,BBB Voice,"
",0,266,1,202744105s,1555487160000000000,XYZ-DMVPN01,CCC File server,"
",0,0,4,202774080hes,1555487160000000000,XYZ-DMVPN01,THEHUB,"
",0,0,4,202774080hes,1555487160000000000,XYZ-DMVPN01,WWW.CISCO.COM,"
",0,248,1,202774103s,1555487220000000000,XYZ-DMVPN01,AAA File server,"
",0,51,1,202774084es,1555487220000000000,XYZ-DMVPN01,BBB File server,"
",0,54,1,202774085es,1555487220000000000,XYZ-DMVPN01,BBB Outlook,"
",0,49,1,202774084es,1555487220000000000,XYZ-DMVPN01,BBB SAP,"
",0,50,1,202774085es,1555487220000000000,XYZ-DMVPN01,BBB SCCM/Backup ,"
",0,52,1,202774084es,1555487220000000000,XYZ-DMVPN01,BBB Tandberg Video,"
",0,46,1,202774084es,1555487220000000000,XYZ-DMVPN01,BBB Voice,"
",0,266,1,202744105s,1555487220000000000,XYZ-DMVPN01,CCC File server,"
",0,0,4,202774080hes,1555487220000000000,XYZ-DMVPN01,THEHUB,"
",0,0,4,202774080hes,1555487220000000000,XYZ-DMVPN01,WWW.CISCO.COM,"
",0,248,1,202774103s,1555487280000000000,XYZ-DMVPN01,AAA File server,"
",0,51,1,202774084es,1555487280000000000,XYZ-DMVPN01,BBB File server,"
",0,54,1,202774085es,1555487280000000000,XYZ-DMVPN01,BBB Outlook,"
",0,49,1,202774084es,1555487280000000000,XYZ-DMVPN01,BBB SAP,"
",0,50,1,202774085es,1555487280000000000,XYZ-DMVPN01,BBB SCCM/Backup ,"
",0,52,1,202774084es,1555487280000000000,XYZ-DMVPN01,BBB Tandberg Video,"
",0,42,1,202804084es,1555487280000000000,XYZ-DMVPN01,BBB Voice,"
",0,266,1,202744105s,1555487280000000000,XYZ-DMVPN01,CCC File server,"
",0,0,4,202804080hes,1555487280000000000,XYZ-DMVPN01,THEHUB,"
",0,0,4,202804080hes,1555487280000000000,XYZ-DMVPN01,WWW.CISCO.COM,"
",0,248,1,202774103s,1555487340000000000,XYZ-DMVPN01,AAA File server,"
",0,51,1,202774084es,1555487340000000000,XYZ-DMVPN01,BBB File server,"
",0,54,1,202774085es,1555487340000000000,XYZ-DMVPN01,BBB Outlook,"
",0,49,1,202774084es,1555487340000000000,XYZ-DMVPN01,BBB SAP,"
",0,50,1,202774085es,1555487340000000000,XYZ-DMVPN01,BBB SCCM/Backup ,"
",0,52,1,202774084es,1555487340000000000,XYZ-DMVPN01,BBB Tandberg Video,"
",0,42,1,202804084es,1555487340000000000,XYZ-DMVPN01,BBB Voice,"
",0,266,1,202744105s,1555487340000000000,XYZ-DMVPN01,CCC File server,"
",0,0,4,202804080hes,1555487340000000000,XYZ-DMVPN01,THEHUB,"
",0,0,4,202804080hes,1555487340000000000,XYZ-DMVPN01,WWW.CISCO.COM,"
",0,248,1,202774103s,1555487400000000000,XYZ-DMVPN01,AAA File server,"
",0,51,1,202774084es,1555487400000000000,XYZ-DMVPN01,BBB File server,"
",0,54,1,202774085es,1555487400000000000,XYZ-DMVPN01,BBB Outlook,"
",0,49,1,202774084es,1555487400000000000,XYZ-DMVPN01,BBB SAP,"
",0,50,1,202774085es,1555487400000000000,XYZ-DMVPN01,BBB SCCM/Backup ,"
",0,52,1,202774084es,1555487400000000000,XYZ-DMVPN01,BBB Tandberg Video,"
",0,42,1,202804084es,1555487400000000000,XYZ-DMVPN01,BBB Voice,"
",0,266,1,202744105s,1555487400000000000,XYZ-DMVPN01,CCC File server,"
",0,0,4,202804080hes,1555487400000000000,XYZ-DMVPN01,THEHUB,"
",0,0,4,202804080hes,1555487400000000000,XYZ-DMVPN01,WWW.CISCO.COM,"
Esben
Hi @sbengo ,
I did a TCPdump of the post command to the influxdb, here is how it looks:
POST /write?consistency=&db=snmpcollector&precision=s&rp=autogen HTTP/1.1
Host: django-dev:8086
User-Agent: snmpcollector
Content-Length: 2608
Authorization: Basic YWRtaWABCDERtaW4=
Content-Type:
Accept-Encoding: gzip
ip_sla_core_switches,hostname=XYZ-DMVPN01,rttMonCtrlAdminTag=CCC\ File\ server rttMonLatestRttOperAddress="
",rttMonLatestRttOperApplSpecificSense=0,rttMonLatestRttOperCompletionTime=253,rttMonLatestRttOperSense=1i,rttMonLatestRttOperTime=202834105 1555487880
ip_sla_core_switches,hostname=XYZ-DMVPN01,rttMonCtrlAdminTag=BBB\ Voice rttMonLatestRttOperAddress="
",rttMonLatestRttOperApplSpecificSense=0,rttMonLatestRttOperCompletionTime=40,rttMonLatestRttOperSense=1i,rttMonLatestRttOperTime=202864084 1555487880
ip_sla_core_switches,hostname=XYZ-DMVPN01,rttMonCtrlAdminTag=BBB\ Outlook rttMonLatestRttOperAddress="
",rttMonLatestRttOperApplSpecificSense=0,rttMonLatestRttOperCompletionTime=51,rttMonLatestRttOperSense=1i,rttMonLatestRttOperTime=202834086 1555487880
ip_sla_core_switches,hostname=XYZ-DMVPN01,rttMonCtrlAdminTag=BBB\ File\ server rttMonLatestRttOperAddress="
",rttMonLatestRttOperApplSpecificSense=0,rttMonLatestRttOperCompletionTime=57,rttMonLatestRttOperSense=1i,rttMonLatestRttOperTime=202834084 1555487880
ip_sla_core_switches,hostname=XYZ-DMVPN01,rttMonCtrlAdminTag=WWW.CISCO.COM rttMonLatestRttOperAddress="
",rttMonLatestRttOperApplSpecificSense=0,rttMonLatestRttOperCompletionTime=0,rttMonLatestRttOperSense=4i,rttMonLatestRttOperTime=202864080 1555487880
ip_sla_core_switches,hostname=XYZ-DMVPN01,rttMonCtrlAdminTag=THEHUB rttMonLatestRttOperAddress="
",rttMonLatestRttOperApplSpecificSense=0,rttMonLatestRttOperCompletionTime=0,rttMonLatestRttOperSense=4i,rttMonLatestRttOperTime=202864080 1555487880
ip_sla_core_switches,hostname=XYZ-DMVPN01,rttMonCtrlAdminTag=BBB\ Tandberg\ Video rttMonLatestRttOperAddress="
",rttMonLatestRttOperApplSpecificSense=0,rttMonLatestRttOperCompletionTime=55,rttMonLatestRttOperSense=1i,rttMonLatestRttOperTime=202834084 1555487880
ip_sla_core_switches,hostname=XYZ-DMVPN01,rttMonCtrlAdminTag=BBB\ SAP rttMonLatestRttOperAddress="
",rttMonLatestRttOperApplSpecificSense=0,rttMonLatestRttOperCompletionTime=54,rttMonLatestRttOperSense=1i,rttMonLatestRttOperTime=202834084 1555487880
ip_sla_core_switches,hostname=XYZ-DMVPN01,rttMonCtrlAdminTag=AAA\ File\ server rttMonLatestRttOperAddress="
",rttMonLatestRttOperApplSpecificSense=0,rttMonLatestRttOperCompletionTime=239,rttMonLatestRttOperSense=1i,rttMonLatestRttOperTime=202834102 1555487880
ip_sla_core_switches,hostname=XYZ-DMVPN01,rttMonCtrlAdminTag=BBB\ SCCM/Backup\ rttMonLatestRttOperAddress="
",rttMonLatestRttOperApplSpecificSense=0,rttMonLatestRttOperCompletionTime=51,rttMonLatestRttOperSense=1i,rttMonLatestRttOperTime=202834086 1555487880
it almost looks like the rttMonLatestRttOperAddress is the one that gives issues.. it sends strange data into the database.
@sbengo , well, it did seem like the multiline input of "rttMonLatestRttOperAddress" was the problem. Now that I do not collect that anymore then data is inserted into the database as expected.
Collecting it via snmpget, it does not print the newlines, but it might be netsnmp that suppresses that?
"ISCO-RTTMON-MIB::rttMonLatestRttOperAddress.100 = STRING: "
"ISCO-RTTMON-MIB::rttMonLatestRttOperAddress.110 = STRING: "
"ISCO-RTTMON-MIB::rttMonLatestRttOperAddress.120 = STRING: "
"ISCO-RTTMON-MIB::rttMonLatestRttOperAddress.130 = STRING: "
"ISCO-RTTMON-MIB::rttMonLatestRttOperAddress.131 = STRING: "
"ISCO-RTTMON-MIB::rttMonLatestRttOperAddress.132 = STRING: "
"ISCO-RTTMON-MIB::rttMonLatestRttOperAddress.140 = STRING: "
"ISCO-RTTMON-MIB::rttMonLatestRttOperAddress.150 = STRING: "
"ISCO-RTTMON-MIB::rttMonLatestRttOperAddress.160 = STRING: "
"ISCO-RTTMON-MIB::rttMonLatestRttOperAddress.170 = STRING: "
Anyways, I have solved my problem as I do not really need that object anyway, but perhaps it would be a good idea for snmpcollector to suppress newlines as it seems to throw influx off course.. Or what do you think?
Thanks for your help and input.
Esben
Hi @hyberdk , thanks for report!
We are quite busy right now, but I agree with you that we need to consider to suppress those new lines.
Can you to open a FR and close this issue?
Thanks again, Regards!
@sbengo , done.. thanks for your help. #405
Please include this information:
0.8.0, commit 3c62474
alpine, in docker
added new snmp metrics, created a new group, added the new group to an existing device. so now it has 2 groups of data to be collected..
to collect the additional group and insert it into the influxdb
I can see in the snmpcollector gui that data is collected correctly, but then I look at the data in the influxdb, time and tags (hostname and rttMonCtrlAdminTag) are missing.
See this output from the database:
the other group is working fine, so only my ip_sla_core_switches have issues.
Its really really strange.. Im a little stuck where to begin.. ;-)
Esben