search

logstash-plugins / logstash-input-snmp

Apache License 2.0
17 stars 22 forks source link

Map Field Names to configurable OID #52

Open coolacid opened 5 years ago

coolacid commented 5 years ago

When using a VPN, SNMP has a tendency to increment a OID index for each connection - My one VPN endpoint is now at index 759. There is a hard limit of 1000 fields in elasticsearch. This means after a number of VPN connections I hit that hard limit.

A solution to this specific scenario (and should probably be setup as configurable for other OIDs with indexes) is to map an OID Alias or Name field as the index value.

Example:

iso.org.dod.internet.mgmt.mib-2.ifMIB.ifMIBObjects.ifXTable.ifXEntry.ifHCInOctets.1

This has an OID Index of 1. Somehow, a configuration should be allowed to replace the one with the value from another OID - In this case:

iso.org.dod.internet.mgmt.mib-2.ifMIB.ifMIBObjects.ifXTable.ifXEntry.ifAlias.1

This would in turn, yield the following for the first example:

iso.org.dod.internet.mgmt.mib-2.ifMIB.ifMIBObjects.ifXTable.ifXEntry.ifHCInOctets.$ValueFromifAlias

It should be noted that ifXEntry has two possible mappings for this - so this needs to be configurable - I'd suggest something along the lines of

` aliasmap => ['iso.org.dod.internet.mgmt.mib-2.ifMIB.ifMIBObjects.ifXTable.ifXEntry.ifHCInOctets', 'iso.org.dod.internet.mgmt.mib-2.ifMIB.ifMIBObjects.ifXTable.ifXEntry.ifAlias', ...]

(Either using the MIB name, or the OID directly)

Care needs to be taken in:

  1. I do not know if Alias can be duplicated, yielding duplicate fields
  2. If a mapping could ever NOT be returned
  3. Should the be cached so lookups don't need to be completed every time

Lots to discuss on this.

coolacid commented 5 years ago

And, this is directly related to #49

axrayn commented 5 years ago

Hi @coolacid,

The 1000 field limit is only a configuration setting and not a strict hard limit.

WRT #49 - The way the SNMP4J library that we're using returns table results doesn't include a row index. The results get returned as an array nested under a field named after the configured table.

I've seen mention of a few occasions where ifAlias/ifName can be duplicated in Cisco kit, but that's normally on things like ATM/ISDN interfaces that have the bearer/signaling/physical 'layers' to them.

Are you talking about setting the index value on walks/tables/both?

colinsurprenant commented 5 years ago

@coolacid also note that we just release v1.1.0 with initial table support (thanks to @axrayn) - maybe look at that and let us know what you think?

coolacid commented 5 years ago

I have to wait for #55 as there is no exact documentation on how to use the new table option.

colinsurprenant commented 5 years ago

@coolacid it is incoming in #57 - can you take a look (asciidoc decoding skillz required ;) and let us know what you think? thanks!

coolacid commented 5 years ago

I finally looked at #45 and see how it's suppose to work. This gets part way to what I'm proposing here.

What I'd suggest is an additional optional configuration to the tables option header which takes a single OID.

This would then change the output of the tables from an array of values, back into objects, but map the ID (final number in the OID) to the text from the header OID.

Thus - if we set header to iso.org.dod.internet.mgmt.mib-2.ifMIB.ifMIBObjects.ifXTable.ifXEntry.ifAlias (which is a table) and table to iso.org.dod.internet.mgmt.mib-2.ifMIB.ifMIBObjects.ifXTable.ifXEntry.ifHCInOctets (which is a table) it would map the text value from ifAlias to the ending of the OID:

If ifAlias.1 == 'eth0' then ifHCInOctets.1 table would be written as ifHCInOctets.eth0 in the object output.