Allow multiple traps per message

[markwalkom]

Currently the snmptrap input expects a single trap per message that is received. However we have users that have devices that spit out multiple traps per message (containing multiple OIDs) and this causes problems as we process it as a single event.

I had a chat to @jordansissel about this and he indicated we should be able to do this reasonably easily without changes to the underlying library.

[jordansissel]

The change here should be pretty easy, instead of emitting 1 event w/ all varbinds as fields, create one event per varbind. The specific code I refer to is here: https://github.com/logstash-plugins/logstash-input-snmptrap/blob/master/lib/logstash/inputs/snmptrap.rb#L75

Any thoughts on the name of the setting that would enable this feature? "one_varbind_per_event" seems a bit long and maybe obtuse.

[allenmchan]

It seems like not all devices send multiple traps per message. So a simple split_traps boolean in the config would be sufficient so the logic doesnt have to go through the trouble of detecting if there are multiple varbinds.

Here is example of a message with multiple traps based from rubydebug output

"message" => "#<SNMP::SNMPv2_Trap:0x1c69b554 @error_index=0, @varbind_list=[#<SNMP::VarBind:0xa8d808f @value=#<SNMP::TimeTicks:0x4ff331b2 @value=61270153>, @name=[1.3.6.1.2.1.1.3.0]>, #<SNMP::VarBind:0x6743c61f @value=[1.3.6.1.4.1.2879.2.10.1.1.5.0.14], @name=[1.3.6.1.6.3.1.1.4.1.0]>, #<SNMP::VarBind:0x12845230 @value=#<SNMP::Gauge32:0x2d2fd6cf @value=4272488803>, @name=[1.3.6.1.4.1.2879.2.8.1.1.1.1.1.0]>, #<SNMP::VarBind:0x1fbfc736 @value=#<SNMP::Integer:0x67afba82 @value=4>, @name=[1.3.6.1.4.1.2879.2.8.1.1.1.1.2.0]>, #<SNMP::VarBind:0x41fde1fc @value=#<SNMP::Gauge32:0x2ff893d @value=442213>, @name=[1.3.6.1.4.1.2879.2.8.1.1.1.1.3.0]>, #<SNMP::VarBind:0x568b0554 @value=\"\a\xDF\x04\b\x14\x1F\n\x00\", @name=[1.3.6.1.4.1.2879.2.8.1.1.1.1.4.0]>, #<SNMP::VarBind:0x410afa2f @value=\"Trunk Group SCL_PEERLESS_PUB has the following automatic control(s) active: callPolicing .\", @name=[1.3.6.1.4.1.2879.2.8.1.1.1.1.5.0]>, #<SNMP::VarBind:0x37af4505 @value=\"SCL_PEERLESS_PUB\", @name=[1.3.6.1.4.1.2879.2.10.1.1.5.1.6.0]>, #<SNMP::VarBind:0x53dc67d2 @value=\"callPolicing \", @name=[1.3.6.1.4.1.2879.2.10.1.1.5.1.14.0]>, #<SNMP::VarBind:0x2b243701 @value=#<SNMP::IpAddress:0x569738c7 @value=\"\n\a\x04f\">, @name=[1.3.6.1.4.1.2879.2.1.7.2.2.4.1.0]>, #<SNMP::VarBind:0x6210bd3a @value=#<SNMP::Integer:0x38f75d22 @value=1>, @name=[1.3.6.1.4.1.2879.2.1.7.2.2.4.3.0]>, #<SNMP::VarBind:0x78038065 @value=\"\n\a\x04f\", @name=[1.3.6.1.4.1.2879.2.1.7.2.2.4.4.0]>], @error_status=0, @request_id=0, @source_ip=\"10.7.17.101\">", "host" => "10.7.17.101", "@version" => "1", "@timestamp" => "2015-04-08T20:31:10.288Z", "tags" => [ [0] "snmptrap"

[jordansissel]

I was preparing for work on this, and I need some clarification ---

The request sounds like it is asking that one event could carry multiple varbinds. If so, that is what the code appears to support today:

https://github.com/logstash-plugins/logstash-input-snmptrap/blob/master/lib/logstash/inputs/snmptrap.rb#L84-L86

The above link is code that adds a field for every varbind present in the received snmptrap.

Is am I reading this request correctly? I originally had interpreted this request to be asking that a single trap with multiple varbinds be able to be split into one event per varbind instead of one event per trap?

Maybe we could show a specific example of a trap and the desired output (one event or multiple events, and what each event would look like) ?

[allenmchan]

@jordansissel Thanks for looking into this issue. What you are describing is what I am looking for. The idea is to split multiple varbinds in an event into individual fields after which i can use filters to manipulate. As you can see from my above comment, this functionality did not work when i tested it in 2015.
Looking through the closed issues / PRs, i do not see any evidence this feature was added after my testing. Do you want me to re-test just in case it was sneaked in?

[jordansissel]

When I test this last week, each varbind becomes a field in a single event. So this works already. Can you test again?

On Tue, Jan 17, 2017 at 9:59 AM Allen Chan notifications@github.com wrote:

@jordansissel https://github.com/jordansissel

What you are describing is what I am looking for.

The idea is to split multiple varbinds in an event into individual fields after which i can use filters to manipulate.

As you can see from my above comment, this functionality did not work when i tested it in 2015.

Looking through the closed issues / PRs, i do not see any evidence this feature was added after my testing.

Do you want me to re-test just in case it was sneaked in?

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/logstash-plugins/logstash-input-snmptrap/issues/6#issuecomment-273247283, or mute the thread https://github.com/notifications/unsubscribe-auth/AAIC6vpRLcdrm07b88v2ggYO-1u0O4rRks5rTQFrgaJpZM4D_2l3 .

[jordansissel]

Testing a trap on Logstash 5.1.1 and sending multiple varbinds in a single trap:

Logstash:

% bin/logstash -e 'input { snmptrap { } } output { stdout { codec => rubydebug } }'
{
  "@timestamp" => 2017-01-15T04:32:34.809Z,
  "0.2"        => "0",
  "1.2"        => "hello world",
  "host"       => "127.0.0.1",
  "@version"   => "1",
  "message"    => "#<SNMP::SNMPv1_Trap:0x3d0eadc5 @enterprise=[1.3.6.1.4.1.3.1.1], @timestamp=#<SNMP::TimeTicks:0x33c94414 @value=0>, @varbind_list=[#<SNMP::VarBind:0x72f44e2f @name=
[0.2], @value=#<SNMP::Integer:0x3a5ca531 @value=0>>, #<SNMP::VarBind:0x43ccbf8a @name=[1.2], @value=\"hello world\">], @specific_trap=0, @source_ip=\"127.0.0.1\", @agent_addr=#<SNMP:
:IpAddress:0x428e0a28 @value=\"\\xC0\\xA8\\x01~\">, @generic_trap=1>",
  "tags"       => []
}

The above event has two fields that come from the snmptrap that I sent, "0.2" and "1.2"

The best I can tell, this code has been largely unchanged since October 2014, so this feature (multiple varbinds in a single event, with each varbind becoming a separate field in the event) has been available for a long time.

Can you test this please? if it's not working for you, I'd like to figure out why.

[jordansissel]

@allenmchan your earlier comment that included a sample output was only part of the event, not a full copy of the whole output.

I'm confident this feature is already implemented in Logstash (Since October 2014), so I'm going to close this.

We can address it separately if this is not working somehow in other scenarios.