While doing very liberal SNMP Walks on a server I came across this issue
Field Names can get exceedingly long and SNMP Walks can create a Huge amount of field names
[2018-11-04T18:58:32,704][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"snmp-2018.11.04", :_type=>"logs", :_routing=>nil}, #<LogStash::Event:0x22bdd815>], :response=>{"index"=>{"_index"=>"snmp-2018.11.04", "_type"=>"logs", "_id"=>"lcAp4WYBIR0n9FiBt-hA", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Limit of mapping depth [20] in index [snmp-2018.11.04] has been exceeded due to object field [iso.org.dod.internet.mgmt.mib-2.dismanEventMIB.dismanEventMIBObjects.mteEvent.mteEventTable.mteEventEntry.mteEventEntryStatus.6.95.115.110.109.112.100.95.109.116.101.84.114]"}}}}
[2018-11-04T18:47:28,169][WARN ][logstash.outputs.elasticsearch] Could not index event to Elasticsearch. {:status=>400, :action=>["index", {:_id=>nil, :_index=>"snmp-2018.11.04", :_type=>"logs", :_routing=>nil}, #<LogStash::Event:0x78eb628b>], :response=>{"index"=>{"_index"=>"snmp-2018.11.04", "_type"=>"logs", "_id"=>"dpof4WYBf0IKXA2OUQXI", "status"=>400, "error"=>{"type"=>"illegal_argument_exception", "reason"=>"Limit of total fields [6000] in index [snmp-2018.11.04] has been exceeded"}}}}
I thought I would report it, though I am sure it will need some thought, on how to deal with very long mibs
I was thinking the following
Try to perform some kind of Grouping
Too Many fields.
I was thinking about logstash-plugins/logstash-input-snmp#24 where I could for "single" documents add an option to convert from.
{
"OID": oidvalue
}
to something like
{
oid_name: "OID"
value: oid_value
}
Not sure how you would do it with one large document
While doing very liberal SNMP Walks on a server I came across this issue
Field Names can get exceedingly long and SNMP Walks can create a Huge amount of field names
I thought I would report it, though I am sure it will need some thought, on how to deal with very long mibs
I was thinking the following
to something like
Not sure how you would do it with one large document