SNMP Plugin: Merging/joining two tables with seperate indexes

[robwdwd]

Feature Request

Pulling data from one table with index into another tables values.

Proposal:

Sometimes one table in SNMP has a field with contains an index value for another SNMP table and it's useful to have the first tables field/values but grab a field or more than one field from another table where an index exists in the first.

Current behaviour:

Two tables can only be merged when the share the same index.

Desired behaviour:

Merge or extract fields from two tables where the indexes are not the same but in one table a field points to the index in the second table.

Use case: [Why is this important (helps with prioritising requests)]

Here is an example use case of collecting CPU statistics of Cisco networking devices. In this case from the first table (1.3.6.1.4.1.9.9.109.1.1.1.1) in that table .1.3.6.1.4.1.9.9.109.1.1.1.1.2 points to an index in the entity table 1.3.6.1.2.1.47.1.1.1.1 and I'm interested in pulling in the value of 1.3.6.1.2.1.47.1.1.1.1.7 which is the text description of the hardware (i.e. the CPU name)

So to pull out the CPU utilisation of 1 and 5 minutes plus the CPU description/name (preferably as a tag) from another table.

Here is an example I tried to use but this does not work as each table is not indexed the same.

  [[inputs.snmp.table]]
    name = "cpuTable"
    inherit_tags = [ "hostname" ]
    index_as_tag = true

    [[inputs.snmp.table.field]]
      name = " cpmCPUTotalPhysicalIndex"
      oid = ".1.3.6.1.4.1.9.9.109.1.1.1.1.2"
      is_tag = true

    [[inputs.snmp.table.field]]
      name = "cpmCPUTotal1minRev"
      oid = ".1.3.6.1.4.1.9.9.109.1.1.1.1.7"

    [[inputs.snmp.table.field]]
      name = "cpmCPUTotal5minRev"
      oid = ".1.3.6.1.4.1.9.9.109.1.1.1.1.8"

    [[inputs.snmp.table.field]]
      name = "entPhysicalName"
      oid = "1.3.6.1.2.1.47.1.1.1.1.7"

This is the index into the entity table where 16733769 is an index there.

snmpwalk -v 2c -c XXXXXXXX router1.example.com ".1.3.6.1.4.1.9.9.109.1.1.1.1.2"
iso.3.6.1.4.1.9.9.109.1.1.1.1.2.34 = INTEGER: 16733769
iso.3.6.1.4.1.9.9.109.1.1.1.1.2.82 = INTEGER: 39967263
iso.3.6.1.4.1.9.9.109.1.1.1.1.2.98 = INTEGER: 3480210
iso.3.6.1.4.1.9.9.109.1.1.1.1.2.514 = INTEGER: 24635790
iso.3.6.1.4.1.9.9.109.1.1.1.1.2.530 = INTEGER: 38114433

one of the CPU utilisation values I want to grab.

snmpwalk -v 2c -c XXXXXXXX router1.example.com ".1.3.6.1.4.1.9.9.109.1.1.1.1.7"
iso.3.6.1.4.1.9.9.109.1.1.1.1.7.34 = Gauge32: 11
iso.3.6.1.4.1.9.9.109.1.1.1.1.7.82 = Gauge32: 2
iso.3.6.1.4.1.9.9.109.1.1.1.1.7.98 = Gauge32: 2
iso.3.6.1.4.1.9.9.109.1.1.1.1.7.514 = Gauge32: 82
iso.3.6.1.4.1.9.9.109.1.1.1.1.7.530 = Gauge32: 7

Looking at the entity table the following index need to be grabbed and merged into the first table and the rest of the table ignored.

snmpwalk -v 2c -c XXXXXXXX router1.example.com "1.3.6.1.2.1.47.1.1.1.1.7" | egrep "16733769|39967263|3480210|24635790|38114433"
iso.3.6.1.2.1.47.1.1.1.1.7.3480210 = STRING: "0/6/* - cpu"
iso.3.6.1.2.1.47.1.1.1.1.7.16733769 = STRING: "0/2/* - host"
iso.3.6.1.2.1.47.1.1.1.1.7.24635790 = STRING: "0/RP0/* - host"
iso.3.6.1.2.1.47.1.1.1.1.7.38114433 = STRING: "0/RP1/* - host"
iso.3.6.1.2.1.47.1.1.1.1.7.39967263 = STRING: "0/5/* - cpu"

[danielnelson]

@phemmer This seems useful to me, do you have any use cases for this feature?

[phemmer]

I've never seen this before. Most unusual. This is what I get for the description on that OID:

# snmptranslate -Td -m all .1.3.6.1.4.1.9.9.109.1.1.1.1.2
CISCO-PROCESS-MIB::cpmCPUTotalPhysicalIndex
cpmCPUTotalPhysicalIndex OBJECT-TYPE
  -- FROM   CISCO-PROCESS-MIB
  -- TEXTUAL CONVENTION EntPhysicalIndexOrZero
  SYNTAX    Integer32 (0..2147483647) 
  MAX-ACCESS    read-only
  STATUS    current
  DESCRIPTION   "The entPhysicalIndex of the physical entity for which
        the CPU statistics in this entry are maintained.
        The physical entity can be a CPU chip, a group of CPUs,
        a CPU card etc. The exact type of this entity is described by
        its entPhysicalVendorType value. If the CPU statistics
        in this entry correspond to more than one physical entity
        (or to no physical entity), or if the entPhysicalTable is
        not supported on the SNMP agent, the value of this object
        must be zero."
::= { iso(1) org(3) dod(6) internet(1) private(4) enterprises(1) cisco(9) ciscoMgmt(9) ciscoProcessMIB(109) ciscoProcessMIBObjects(1) cpmCPU(1) cpmCPUTotalTable(1) cpmCPUTotalEntry(1) 2 }

Though I think this should be easy to implement in the code. This is my first thought on what config might look like:

    [[inputs.snmp.table.join]]
      index_map_oid = ".1.3.6.1.4.1.9.9.109.1.1.1.1.2"
      foreign_table_oid = "1.3.6.1.2.1.47.1.1.1.1.7"

    [[inputs.snmp.table.field]]
      name = "cpmCPUTotal1minRev"
      oid = ".1.3.6.1.4.1.9.9.109.1.1.1.1.7"

    [[inputs.snmp.table.field]]
      name = "entPhysicalName"
      oid = "1.3.6.1.2.1.47.1.1.1.1.7"
      is_tag = true

The code would process the table.join first, build the mapping of main table index to foreign table index, and then process each of the table.field definitions. When the oid of a table.field falls under the table.join.foreign_table_oid, instead of using the index of the foreign table, it would use the mapping and translate the foreign index to the main index.

[robwdwd]

@phemmer that sounds like a good solution. Although one thing to note is that the second (foreign) table in this case has a lot more entries than we want. All that is needed is the entries matching the index in the first table and the rest can be discarded.

It is quite unusual but Cisco seem to like using it unfortunately. I think they do it with linecard memory as well. I can't recall off the top of my head but I think some other network vendors do similar things.

[puckpuck]

I'm working on a similar situation with another prospect. They previously solved this with a custom python script which was hectic. In there case however they could have hundreds of rows in their tables, some tables have 5 indexes, others have 3, and the order you join them isn't straight forward. I contemplated using regex expression to specify the joins.

This is from an Alcatel 7750 router Get point tags for a description/alias from a table that has an index made up of: svcId, sapPortId, sapEncapValue Get values from another table that has an index made up of: tmnxSubInfoSubIdent, sapPortId, sapEncapValue, tmnxSLAProfName, tmnxSPIEgrQStatsQueueId

In this case you can line up sapPortId, and sapEncapValue but not the others. In this particular implementation, the table with metrics will have the first index and fourth index as always being the same, so they could be ignored, and the last item (tmnxSPIEgrQStatsQueueId) actually lines up with the first item from the point tags table (svcId).

My thought was to use regex for index matching instead. So you can have a property that defined with the metric that says this is how you join me back to the point tags table: \..*\.$2\.$3\..*\.$1

I'm open to any ideas really to tackle this. My approach is to build the point tags first, and load them into a map, then join the data form the 'metrics' tables and join them in for point tags.

[[inputs.snmp.tags]]
  oid = "TABLE_WITH_TAGS_OID"
  name = "give this a unique name referenced later"

  [[inputs.snmp.tags.field]]
    oid = "FIELD_OID"
    name = "point tag name"

...

[[inputs.snmp.table]]
  oid = "TABLE_WITH_METRICS_OID"

  [[inputs.snmp.table.tagjoin]]
     name = "name given to tags from above"
     index_regex = "REGEX pattern to match index on"

  [[inputs.snmp.field]]
    oid = "FIELD_OID"
    name = "metric name"

...

[puckpuck]

The regex could also be done the other way... whatever is simpler really, but since indexes are dot delimited, it's easy to create a pattern for it and expose $1 .. $n regex groups that are used to match

[scollora]

I have a use case where this would be incredibly useful. I have a Cisco WLC where I need a join of the APTable and the ClientTable so I can map a client to an APGroup. The key/index of the APTable is APMAC and APMAC is contained in the ClientTable, so it would be nice to have that mapping.

[Foggy2]

This would also be very useful for me. Using Cisco equipment and many of the tables reference other tables with alternative index values that are stored under one of the OIDs.

It would also be very useful if you can link to more than one foreign table/index as I have seen some complex OID/table structures when trying to query for specific information.

I don't know any GoLang but I am happy to try and help where I can to get this functionality implemented. Just let me know if there is something I can assist with.

[bmagistro]

Another use case (might be covered/supported) is joining two interface tables "IF-MIB::ifTable" and "IF-MIB::ifXTable". In our case it looks like ifTable::ifDescr can be used as a tag that would match up with ifXTable::ifName. End goal would be able to use ifName (tag?) and get all info for the interface from both tables.

EDIT: Looks like this might be working as is in 1.7.0, by setting ifName to the appropriate oid for each table and having the tables be named the same

[[inputs.snmp.table]]
  name = "interface"
  oid = "IF-MIB::ifTable"
  inherit_tags = [ "hostname" ]
[[inputs.snmp.table.field]]
  name = "ifName"
  oid = "IF-MIB::ifDescr"
  is_tag = true
[[inputs.snmp.table]]
  name = "interface"
  oid = "IF-MIB::ifXTable"
  inherit_tags = [ "hostname" ]
[[inputs.snmp.table.field]]
  name = "ifName"
  oid = "IF-MIB::ifName"
  is_tag = true

[marxin]

I have one another use case. Mikrotik SNMP provides list of wireless registrations (/interface wireless registration-table), where each entry contains index to a different SNMP table with list of interfaces (/interface). That's currently the only way one can get Interface name of a registered client.

[nward]

I've never seen this before. Most unusual.

This is common in SNMP. This is commonly used with ifIndex as described above, and the entity MIB is used for identifying physical components in many many vendors. There are many other examples of this, but, these are the most commonly used instances of this.

[helotpl]

I have a semi working solution here: https://github.com/helotpl/telegraf/commit/627fcf3ad8837a10701254f3b1e6f965434110e7

Still working on adding capability to perform outer join on such tables. Basically it adds following options to input.snmp: secondary_index_table = true/false - specifies a field to be use as translation index, for example ADVA-FSPR7-MIB::entityFacilityOneIndex can be used here.

secondary_index_use = true/false - specifies if this field uses different index than current table, and uses table from secondary_index_table to join it into primary table

not implemented yet: secondary_outer_join = true/false - controls what happens if index from secondary table is not found in secondary_index_table, should value be added or dropped

[helotpl]

  [[inputs.snmp.table]]
    name = "cpuTable"
    inherit_tags = [ "hostname" ]
    index_as_tag = true

    [[inputs.snmp.table.field]]
      name = " cpmCPUTotalPhysicalIndex"
      oid = ".1.3.6.1.4.1.9.9.109.1.1.1.1.2"
      is_tag = true
      secondary_index_table = true

    [[inputs.snmp.table.field]]
      name = "cpmCPUTotal1minRev"
      oid = ".1.3.6.1.4.1.9.9.109.1.1.1.1.7"

    [[inputs.snmp.table.field]]
      name = "cpmCPUTotal5minRev"
      oid = ".1.3.6.1.4.1.9.9.109.1.1.1.1.8"

    [[inputs.snmp.table.field]]
      name = "entPhysicalName"
      oid = "1.3.6.1.2.1.47.1.1.1.1.7"
      secondary_index_use = true

example output for this config:

2020-05-07T11:23:45Z I! Starting Telegraf
> cpuTable,\ cpmCPUTotalPhysicalIndex=11000,agent_host=X.X.X.X,host=X.X.X.X,index=11000 cpmCPUTotal1minRev=8i,cpmCPUTotal5minRev=8i,entPhysicalName="Switch2 Supervisor 1 (virtual slot 11)" 1588850628000000000
> cpuTable,\ cpmCPUTotalPhysicalIndex=1000,agent_host=X.X.X.X,host=X.X.X.X,index=1000 cpmCPUTotal1minRev=5i,cpmCPUTotal5minRev=5i,entPhysicalName="Switch1 Supervisor 1 (virtual slot 1)" 1588850628000000000

I have no cisco router here, and on switches these two tables have same indexes.

[afletch]

Another use case (might be covered/supported) is joining two interface tables "IF-MIB::ifTable" and "IF-MIB::ifXTable". In our case it looks like ifTable::ifDescr can be used as a tag that would match up with ifXTable::ifName. End goal would be able to use ifName (tag?) and get all info for the interface from both tables.

EDIT: Looks like this might be working as is in 1.7.0, by setting ifName to the appropriate oid for each table and having the tables be named the same

[[inputs.snmp.table]]
  name = "interface"
  oid = "IF-MIB::ifTable"
  inherit_tags = [ "hostname" ]
[[inputs.snmp.table.field]]
  name = "ifName"
  oid = "IF-MIB::ifDescr"
  is_tag = true
[[inputs.snmp.table]]
  name = "interface"
  oid = "IF-MIB::ifXTable"
  inherit_tags = [ "hostname" ]
[[inputs.snmp.table.field]]
  name = "ifName"
  oid = "IF-MIB::ifName"
  is_tag = true

Are you sure this is working, or is it simply returning both tables separately (what do you see, if you run with telegraf on the cli with --debug --test) ?

[helotpl]

This will not work, from toml loading the second table will overwrite the first one since thy have same name. You need to define first table and add one by one fields from second table to it. They share same index so it will work. In essence result will be exactly same, because when supplying oid to inputs.snmp.table the plugin calls snmptranslate and adds fields based on mibs, not by walking whole table (examine the source to confirm this, this happens in: func snmpTableCall(oid string) (mibName string, oidNum string, oidText string, fields []Field, err error) { )

[danielnelson]

This will not work, from toml loading the second table will overwrite the first one since thy have same name.

Tables should never be overwritten, although you might trigger an error. Recall that double bracket [[tables]] are an array of tables, and can be specified multiple times.

I suspect this works but gives multiple lines of output. If the tags match it will be merged on the database side.

I think we are only willing to add join code to the snmp input if it is very simple and doesn't significantly increase config complexity. It probably makes more sense to make this an aggregator plugin.

[helotpl]

I think we are only willing to add join code to the snmp input if it is very simple and doesn't significantly increase config complexity. It probably makes more sense to make this an aggregator plugin.

For me it also doesn't make sense to join tables at input stage. But in my solution join happens when data is still at raw stage, at reading values from ifv map[string]string storing "index" to "value" translation from snmp. Here no tags are yet applied, so when you know which column needs translation than you simply perform to secondary index (second table). Only downside of this solution is that primary and secondary index are determined by translation table. If you have one translation column then you cannot choose which index is primary and which is secondary.

Once table is populated, when you try to join two tables, then you have to perform expensive comparisons of all tags between tables. Even worse, some tags might not be present in secondary table, then you will have supply a config that permits selection of tags (you can just join based on index, because you have to select hostname if you pull data from several switches for example).

I have this solution working for several days in my production environment, and it is working perfectly so far.

One more thing to add, I need sometimes to add tags and no fields between joined tables. You cannot in current scheme collect just tags for index. In my previous solution I had to store tags as fields before join in kapacitor, then switch them to tags. This solution was killing my server, because join buffers unjoined points and this not only fills memory over time, but also increases cpu consumption when all new points are compared against memory full of unjoined points.

[Hipska]

Hi peeps, I missed this conversation, but what do you think of my suggestion mentioned in https://github.com/influxdata/telegraf/issues/7246#issuecomment-656291117 ?

[helotpl]

I think this was incorrectly merged. This post is about joining two tables which have two different indexes, and lookup table that merges them. For example in Cisco you have entity table and iftable, both have different indexes, but they can be merged.

[Hipska]

Correct, read my post again and the previous posts from me in that topic.

[LeadingMoominExpert]

I think I'm coming close to solving this by using this feature: https://github.com/influxdata/telegraf/pull/8466. With the ability to translate OID-values and using the regex processor plugin it's possible to yield these aliases as integers. I used ENTITY-MIB::entPhysicalTable and IF-MIB::ifXTable for example. Relevant configuration:

[[processors.regex]]
  namepass = ["entPhysicalTable"]

  [[processors.regex.tags]]
    key = "aliasMapping"
    pattern = "[^0-9]+"
    replacement = ""

[[inputs.snmp.table]]
    oid = "ENTITY-MIB::entPhysicalTable"
    index_as_tag = true

    [[inputs.snmp.table.field]]
      name = "aliasMapping"
      oid = "ENTITY-MIB::entAliasMappingIdentifier"
      oid_index_suffix = ".0"
      is_tag = true
      translate = true

[[inputs.snmp.table]]
    name = "ifXTable"
    ##oid = "IF-MIB::ifXTable"
    index_as_tag = true

With these any two entries that can be merged together have nearly identical tags, e.g. entPhysicalTable tags:

"tags":{"agent_host":"10.10.10.10","aliasMapping":"10106","index":"1015"},"timestamp":1607016093

ifXTable tags:

"tags":{"agent_host":"10.10.10.10","index":"10106"},"timestamp":1607016093

These could be merged together with the aliasMapping in the first, with respect to the index in the latter. Unfortunately the merge aggregator wouldn't merge these as is, since it requires the same tag set.

For the purpose of joining two (or more) snmp-tables like this, it should be straightforward to implement a separate aggregator.

[Dave-Day1]

I am working on a situation similar to the above scenarios, only I am having to try and join three tables to get the information I want. However, one table doesn't really have a field I want but does have an index field that matches the first table and an index that matches the index of the third table. I am wanting to pull Power-over-Ethernet status from the third table in the list and add the Port Name to it from the first table in the list. I am using the following tables:

• ENTITY-MIB::entPhysicalTable
• CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortTable
• POWER-ETHERNET-MIB::pethPsePortTable

The ENTITY-MIB::entPhysicalTable contains the entPhysicalName field, such as GigabitEthernet1/0/1 and the Index value of 1010 and I want to match that to the pethPsePortDetectionStatus in the POWER-ETHERNET-MIB::pethPsePortTable. However, the POWER-ETHERNET-MIB::pethPsePortTable has index values of 1.1 for what would be GigabitEthernet1/0/1. The issue is, as you can see, the Index values do not match 1010 --- 1.1.

This is why I am trying to use a third table, CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortTable, to connect the two fields I want. The CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortTable has an index value that matches the index values of POWER-ETHERNET-MIB::pethPsePortTable and a separate field, cpeExtPsePortEntPhyIndex, that contains the index value from the ENTITY-MIB::entPhysicalTable.

The only two fields I really need is the ENTITY-MIB::entPhysicalName and the ETHERNET-MIB::pethPsePortDetectionStatus.

Here is what I have tried:

Power-Over-Ethernet status

[[inputs.snmp.table]] name = "CISCO_POE" oid = "CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortTable" index_as_tag = true

[[inputs.snmp.table.field]]
  name = "Ent_Index"
  oid = "CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortEntPhyIndex"
  is_tag = true
  seconday_index_table = true

[[inputs.snmp.table.field]]
  name = "Status"
  oid = "POWER-ETHERNET-MIB::pethPsePortDetectionStatus"

[[inputs.snmp.table.field]]
  name = "Port"
  oid = "ENTITY-MIB::entPhysicalName"
  secondary_index_use = true

It doesn't work with the secondary_index and if I comment them out I get the first two tables to join but don't get the "Port" field.

Any suggestions? Thanks.

[Dave-Day1]

So here is where I am at now. I am using the below code:

Power-Over-Ethernet status

[[inputs.snmp.table]] name = "CISCO-POE" index_as_tag = true

[[inputs.snmp.table.field]]
  name = "Ent_Index"
  oid = "CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortEntPhyIndex"
  is_tag = true

[[inputs.snmp.table.field]]
  name = "Status"
  oid = "POWER-ETHERNET-MIB::pethPsePortDetectionStatus"

[[inputs.snmp.table]] name = "Port" inherit_tags = [ " Ent_Index " ] index_as_tag = true

[[inputs.snmp.table.field]]
  name = "Port_Name"
  oid = "ENTITY-MIB::entPhysicalName"

The following is a sample of the output:

CISCO-POE,_EntIndex=1010,agent_host=10.252.192.10,host=CDC1SLGCOLC,index=1.1 Status=2i 1620161350000000000 CISCO-POE,_EntIndex=1011,agent_host=10.252.192.10,host=CDC1SLGCOLC,index=1.2 Status=2i 1620161350000000000 CISCO-POE,_EntIndex=1012,agent_host=10.252.192.10,host=CDC1SLGCOLC,index=1.3 Status=2i 1620161350000000000 CISCO-POE,_EntIndex=1013,agent_host=10.252.192.10,host=CDC1SLGCOLC,index=1.4 Status=2i 1620161350000000000 CISCO-POE,_EntIndex=1014,agent_host=10.252.192.10,host=CDC1SLGCOLC,index=1.5 Status=2i 1620161350000000000 CISCO-POE,_EntIndex=1015,agent_host=10.252.192.10,host=CDC1SLGCOLC,index=1.6 Status=2i 1620161350000000000 Port,agent_host=10.252.192.10,host=CDC1SLGCOLC,index=1010 Port_Name="GigabitEthernet1/0/1" 1620161350000000000 Port,agent_host=10.252.192.10,host=CDC1SLGCOLC,index=1011 Port_Name="GigabitEthernet1/0/2" 1620161350000000000 Port,agent_host=10.252.192.10,host=CDC1SLGCOLC,index=1012 Port_Name="GigabitEthernet1/0/3" 1620161350000000000 Port,agent_host=10.252.192.10,host=CDC1SLGCOLC,index=1013 Port_Name="GigabitEthernet1/0/4" 1620161350000000000 Port,agent_host=10.252.192.10,host=CDC1SLGCOLC,index=1014 Port_Name="GigabitEthernet1/0/5" 1620161350000000000 Port,agent_host=10.252.192.10,host=CDC1SLGCOLC,index=1015 Port_Name="GigabitEthernet1/0/6" 1620161350000000000

What I need to do is be able to tie the Index next to Port_Name to the _EntIndex so that I get the Port_Name and Status.

Thanks.

[helotpl]

Hi Dave, Are you using https://github.com/helotpl/telegraf for this configuration? My changes were never added to main in telegraf. kind regards, Marek

[Dave-Day1]

I downloaded telegraf through the InfluxDB website. I am using version Telegraf 1.18.1 (git: HEAD d3ac9a3f). I am fairly new at getting things off of GitHub and adding it to what I have installed. If I could get some instructions or an assist on how to get this done I would greatly appreciate it. Thanks.

[Hipska]

@Dave-Day1 you can try it out by using one of the artefacts mentioned here.

[helotpl]

I would like to ask any subscribers of this issue, are you in need to join more than two tables? There is a possibility to find such example, but does someone really need it?

[instabin]

@hipska @Dave-Day1

I am trying to get POE status as well on a cisco switch. I am trying to merge it with the data pulled from IF-MIB for the interface stats current data rate / dropped pkts ect. I would like to pull in the power usage from the CISCO-POWER-ETHERNET-EXT-MIB

Has anyone made any progress or have some config examples?

[helotpl]

@instabin is there translation table to ifTable?

[instabin]

@helotpl Im do not have access to the switch currently but if I remember correctly The Physical interface has index with the interface id's

The poe has column with the physical index number listed.

[helotpl]

@instabin like @Dave-Day1 wrote, CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortEntPhyIndex will move you to ENTITY-MIB to it's ENTITY-MIB::entPhysicalTable. Is there transition to IF-MIB::ifTable index?

[helotpl]

There is transition from ENTITY-MIB to IF-MIB. This transition field is ENTITY-MIB::entAliasMappingIdentifier. It needs striping .0 from entity index. So far patch that I was working on allows only to join two tables, not three.

[instabin]

So In the EntityMib entPhysicalTable there is entPhysicalAlias which is the index value of the iftable.

In the ciscoPowerEthernetExtMIB there is cpeExtPsePortEnthPhyIndex which is the index value of the entPhysicalTable.

[helotpl]

Like I wrote earlier, patch mentioned here works on two tables not three. You can still pull interface name from entPhysicalTable like @Dave-Day1 mentioned earlier, and then you can display per interface stats in graphana by using variables.

[instabin]

This will work as long as uncomment index_as_tag = true

if index_as_tag is commented out telegraf returns this

ciscoPOEPort,agent_host=172.20.0.1,host=NMS,sysName=MDF-Stack DeviceDetected=2i,EntPhyIndex=5024i,PortPwrConsumption=0i 1621265588000000000

if index_as tag is uncommented I get.

ciscoPOEPort,EntPhysicalName=GigabitEthernet5/0/30,agent_host=172.20.0.1,host=NMS,index=5.30,sysName=MDF-Stack DeviceDetected=1i,EntPhyIndex=5035i,PortPwrConsumption=1591i 1621266112000000000

POE Stats

[[inputs.snmp.table]] name = "ciscoPOEPort" inherit_tags = [ "sysName","sysLocation" ]

index_as_tag = true

[[inputs.snmp.table.field]]
name = "DeviceDetected"
oid = "CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortDeviceDetected"

[[inputs.snmp.table.field]]
name = "PortPwrConsumption"
oid = "CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortPwrConsumption"

[[inputs.snmp.table.field]]
name = "EntPhyIndex"
oid = "CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortEntPhyIndex"
secondary_index_table = true

[[inputs.snmp.table.field]]
name = "EntPhysicalName"
oid = "ENTITY-MIB::entPhysicalName"
secondary_index_use = true
is_tag = true

[helotpl]

Are you using telegraf with this patch? https://github.com/influxdata/telegraf/pull/9241#issuecomment-836467790 These changes have not yet been accepted to telegraf main.

[instabin]

Yes I was using the one you linked to earlier in this thread.

here is the link https://github.com/influxdata/telegraf/pull/9241#issuecomment-836467790 Also it seems like the inherit_tags = [ "sysName","sysLocation" ] is not working, only on the poe section where the tables are merged.

Here is my full config

[[inputs.snmp]] agents = [ "172.20.0.1","172.20.0.4","172.20.0.7" ] version = 2 community = "mysnmpcommunity" [[inputs.snmp.field]] name = "sysName" oid = "SNMPv2-MIB::sysName.0" is_tag = true [[inputs.snmp.field]] name = "sysLocation" oid = "SNMPv2-MIB::sysLocation.0" is_tag = true [[inputs.snmp.field]] name = "uptime" oid = "DISMAN-EVENT-MIB::sysUpTimeInstance" [[inputs.snmp.field]] name = "ciscoProduct" oid = "iso.3.6.1.4.1.9.9.402.1.3.1.2.1" [[inputs.snmp.field]] name = "ciscoSoftware" oid = "iso.3.6.1.4.1.9.9.500.1.2.1.1.8.1001"

Temps

[[inputs.snmp.table]] name = "ciscoEnvMonTemperatureStatusTable" inherit_tags = ["sysName","sysLocation"] [[inputs.snmp.table.field]] name = "ciscoEnvMonTemperatureStatusDescr" oid = "iso.3.6.1.4.1.9.9.13.1.3.1.2" [[inputs.snmp.table.field]] name = "ciscoEnvMonTemperatureStatusValue" oid = "iso.3.6.1.4.1.9.9.13.1.3.1.3" [[inputs.snmp.table.field]] name = "ciscoEnvMonTemperatureThreshold" oid = "iso.3.6.1.4.1.9.9.13.1.3.1.4" [[inputs.snmp.table.field]] name = "ciscoEnvMonTemperatureLastShutdown" oid = "iso.3.6.1.4.1.9.9.13.1.3.1.5" [[inputs.snmp.table.field]] name = "ciscoEnvMonTemperatureState" oid = "iso.3.6.1.4.1.9.9.13.1.3.1.6"

Powersupply

[[inputs.snmp.table]] name = "ciscoEnvMonSupplyStatusTable" inherit_tags = ["sysName","sysLocation"] [[inputs.snmp.table.field]] name = "ciscoEnvMonSupplyStatusDescr" oid = "iso.3.6.1.4.1.9.9.13.1.5.1.2" [[inputs.snmp.table.field]] name = "ciscoEnvMonSupplyState" oid = "iso.3.6.1.4.1.9.9.13.1.5.1.3" [[inputs.snmp.table.field]] name = "RciscoEnvMonSupplySource" oid = "iso.3.6.1.4.1.9.9.13.1.5.1.4"

[[inputs.snmp.table]] name = "ciscoCPU" inherit_tags = ["sysName","sysLocation"] [[inputs.snmp.table.field]] name = "cpmCPUTotal5secRev" oid = ".1.3.6.1.4.1.9.9.109.1.1.1.1.6" [[inputs.snmp.table.field]] name = "cpmCPUTotal1minRev" oid = ".1.3.6.1.4.1.9.9.109.1.1.1.1.7" [[inputs.snmp.table.field]] name = "cpmCPUTotal5minRev" oid = ".1.3.6.1.4.1.9.9.109.1.1.1.1.8"

Interface Stats

[[inputs.snmp.table]] name = "ciscoInterface" inherit_tags = [ "sysName","sysLocation" ]

[[inputs.snmp.table.field]]
  oid = "IF-MIB::ifHCInOctets"

[[inputs.snmp.table.field]]
  oid = "IF-MIB::ifHCOutOctets"

[[inputs.snmp.table.field]]
  oid = "IF-MIB::ifDescr"
  is_tag = true

[[inputs.snmp.table.field]]
  oid = "IF-MIB::ifAlias"
  is_tag = true

[[inputs.snmp.table.field]]
  oid = "IF-MIB::ifHighSpeed"

[[inputs.snmp.table.field]]
  oid = "IF-MIB::ifOperStatus"

[[inputs.snmp.table.field]]
  oid = "IF-MIB::ifLastChange"

[[inputs.snmp.table.field]]
  oid = "IF-MIB::ifInErrors"

[[inputs.snmp.table.field]]
  oid = "IF-MIB::ifOutErrors"

[[inputs.snmp.table.field]]
  oid = "IF-MIB::ifInDiscards"

[[inputs.snmp.table.field]]
  oid = "IF-MIB::ifOutDiscards"

POE Stats

[[inputs.snmp.table]] name = "ciscoPOEPort" inherit_tags = [ "sysName","sysLocation" ] index_as_tag = true

[[inputs.snmp.table.field]]
name = "DeviceDetected"
oid = "CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortDeviceDetected"

[[inputs.snmp.table.field]]
name = "PortPwrConsumption"
oid = "CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortPwrConsumption"

[[inputs.snmp.table.field]]
name = "EntPhyIndex"
oid = "CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortEntPhyIndex"
secondary_index_table = true

[[inputs.snmp.table.field]]
name = "EntPhysicalName"
oid = "ENTITY-MIB::entPhysicalName"
secondary_index_use = true
is_tag = true

[helotpl]

I tested your config and it works:

Config:

[[inputs.snmp]]
agents = ["X.X.X.X"]
version = 2
community = "XXXXXXXX"

[[inputs.snmp.table]]
name = "ciscoPOEPort"
#inherit_tags = [ "sysName","sysLocation" ]
index_as_tag = true

[[inputs.snmp.table.field]]
name = "DeviceDetected"
oid = "CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortDeviceDetected"

[[inputs.snmp.table.field]]
name = "PortPwrConsumption"
oid = "CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortPwrConsumption"

[[inputs.snmp.table.field]]
name = "EntPhyIndex"
oid = "CISCO-POWER-ETHERNET-EXT-MIB::cpeExtPsePortEntPhyIndex"
secondary_index_table = true

[[inputs.snmp.table.field]]
name = "EntPhysicalName"
oid = "ENTITY-MIB::entPhysicalName"
secondary_index_use = true
is_tag = true

Result:

# test/telegraf/telegraf --config test2.conf --test
2021-05-18T13:17:32Z I! Starting Telegraf
> ciscoPOEPort,EntPhysicalName=GigabitEthernet1/4,agent_host=X.X.X.X,host=xxx,index=1.4 DeviceDetected=1i,EntPhyIndex=1004i,PortPwrConsumption=8143i 1621343853000000000
> ciscoPOEPort,EntPhysicalName=GigabitEthernet1/23,agent_host=X.X.X.X,host=xxx,index=1.23 DeviceDetected=2i,EntPhyIndex=1023i,PortPwrConsumption=0i 1621343853000000000
> ciscoPOEPort,EntPhysicalName=GigabitEthernet1/45,agent_host=X.X.X.X,host=xxx,index=1.45 DeviceDetected=2i,EntPhyIndex=1045i,PortPwrConsumption=0i 1621343853000000000
> ciscoPOEPort,EntPhysicalName=GigabitEthernet1/35,agent_host=X.X.X.X,host=xxx,index=1.35 DeviceDetected=2i,EntPhyIndex=1035i,PortPwrConsumption=0i 1621343853000000000

[instabin]

Did you have any issue with it when index_as_tag is commented out? Im going to drop the measurement and re-add it.

POE Stats

[[inputs.snmp.table]] name = "ciscoPOEPort" inherit_tags = [ "sysName","sysLocation" ]

index_as_tag = true

[helotpl]

Apparently it works with index_as_tag = true at table level, and doesn't work when it is set to false.

index_as_tag = true:

# test/telegraf/telegraf --config test2.conf --test | grep -E "1005|1003"
2021-05-19T07:13:35Z I! Starting Telegraf
> ciscoPOEPort,EntPhysicalName=GigabitEthernet1/5,agent_host=X.X.X.X,host=xxx,index=1.5 DeviceDetected=1i,EntPhyIndex=1005i,PortPwrConsumption=8143i 1621408416000000000
> ciscoPOEPort,EntPhysicalName=GigabitEthernet1/3,agent_host=X.X.X.X,host=xxx,index=1.3 DeviceDetected=1i,EntPhyIndex=1003i,PortPwrConsumption=6858i 1621408416000000000

index_as_tag = false

# test/telegraf/telegraf --config test2.conf --test | grep -E "1005|1003"
2021-05-19T07:13:59Z I! Starting Telegraf
> ciscoPOEPort,agent_host=X.X.X.X,host=xxx DeviceDetected=1i,EntPhyIndex=1005i,PortPwrConsumption=8143i 1621408440000000000
> ciscoPOEPort,agent_host=X.X.X.X,host=xxx DeviceDetected=1i,EntPhyIndex=1003i,PortPwrConsumption=6858i 1621408440000000000

I'll look into this why this happens.

[helotpl]

I have it working now. Just need to add some unit tests for this, and soon will release an update for the patch. Outputs with and without index_as_tag:

% ./telegraf --config test2.conf --test | grep -E "1005|1003"
2021-05-19T08:53:06Z I! Starting Telegraf
> ciscoPOEPort,EntPhysicalName=GigabitEthernet1/5,agent_host=X.X.X.X,host=xxx DeviceDetected=1i,EntPhyIndex=1005i,PortPwrConsumption=8143i 1621414387000000000
> ciscoPOEPort,EntPhysicalName=GigabitEthernet1/3,agent_host=X.X.X.X,host=xxx DeviceDetected=1i,EntPhyIndex=1003i,PortPwrConsumption=6858i 1621414387000000000
% ./telegraf --config test2.conf --test | grep -E "1005|1003"
2021-05-19T08:53:19Z I! Starting Telegraf
> ciscoPOEPort,EntPhysicalName=GigabitEthernet1/5,agent_host=X.X.X.X,host=xxx,index=1.5 DeviceDetected=1i,EntPhyIndex=1005i,PortPwrConsumption=8143i 1621414400000000000
> ciscoPOEPort,EntPhysicalName=GigabitEthernet1/3,agent_host=X.X.X.X,host=xxx,index=1.3 DeviceDetected=1i,EntPhyIndex=1003i,PortPwrConsumption=6858i 1621414400000000000

[helotpl]

Correction is live now. If this patch works for you and you think that it solves your problem, than please write it here. So maybe it will be accepted to telegraf main.

[instabin]

I downloaded and installed the new artifacts. They do work with the index_as_tag = false

Is there anything else I could do to help these changes get accepted to the telegraf main?

[helotpl]

Is there anything else I could do to help these changes get accepted to the telegraf main?

To tell the truth I don't know. Apparently you are the only one that publicly admitted to usage of my changes. Pull request is rejected because of some cosmetic changes in documentation, and now it is waiting for reply for almost a week. I'm close to giving up, because I have a working solution for my problem, and apparently nobody else is suffering from it.

[Hipska]

Relax, I'm a volunteer community member, so I need to find some free time to process. Also, your request is not rejected, it is still open and waiting to be merged. It will be merged soon, just be patient, it is only 20 days open, we have others that are open for way longer.

[msrpub]

Is this planned to be meget into main soon, or there was some issue preventing it? Have several usecases waiting for this as well

[Hipska]

It seems @helotpl has closed his PR by himself.

[msrpub]

Anything else similar to make joins? In promql I can make them in query language, but in influxql I'm not aware of a way to accomplish the same.

[helotpl]

It seems @helotpl has closed his PR by himself.

Well i would expect that grown up person would write when he is not going to accept PR. Your approach to make me work on unit tests and docs for the changes and after that stop replying is not serious at all. I know that end result configuration is not what you would like. BUT you should write that from the start! Not force me to do additional work for nothing!

[Hipska]

Again, I'm just a community volunteer and trying to triage PR's doing a best effort approach to you get it accepted. I have indeed requested changes and accepted the PR as soon as you applied them. I marked the PR as ready for final review by Influx staff, so I did all I could. Please don't go personal, I wasn't the one that closed the PR. With the PR being closed, that means the author doesn't want it to be merged anymore or lost interest. If you do want it to get merged, please reopen.

[Hipska]

@msrpub If you want to get the interface name, you can use the ifName processor. For other kinds of merges/lookups there is no alternative in telegraf yet. (Yes, also need/want such kind of possibility in telegraf)