etingof / pysnmp

Python SNMP library
http://snmplabs.com/pysnmp/
BSD 2-Clause "Simplified" License
576 stars 197 forks source link

PysnmpError with dispatch.py(49) #15

Open leonhoffman opened 8 years ago

leonhoffman commented 8 years ago

1.) When just calling something like 'sysDescr' in the SNMPv2-MIB I get results.

#!/usr/bin/env python

from pysnmp.hlapi import *
from pprint import pprint

community = 'communitypassword'
lb = 'f5ltm-host'

g = nextCmd(SnmpEngine(),
            CommunityData(community),
            UdpTransportTarget((lb, 161)),
            ContextData(),
            ObjectType(ObjectIdentity('SNMPv2-MIB', 'sysDescr')))

print next(g)

Produces:

(None, Integer('noError', NamedValues(('noError', 0), ('tooBig', 1), ('noSuchName', 2), ('badValue', 3), ('readOnly', 4), ('genErr', 5), ('noAccess', 6), ('wrongType', 7), ('wrongLength', 8), ('wrongEncoding', 9), ('wrongValue', 10), ('noCreation', 11), ('inconsistentValue', 12), ('resourceUnavailable', 13), ('commitFailed', 14), ('undoFailed', 15), ('authorizationError', 16), ('notWritable', 17), ('inconsistentName', 18))), Integer(0, subtypeSpec=ConstraintsIntersection(ConstraintsIntersection(), ValueRangeConstraint(0, Integer(2147483647)))), [ObjectType(ObjectIdentity(ObjectName('1.3.6.1.2.1.1.1.0')), DisplayString('BIG-IP 1600 : Linux 2.6.32-431.56.1.el6.f5.x86_64 : BIG-IP software release 12.1.0, build 1.0.1447', subtypeSpec=ConstraintsIntersection(ConstraintsIntersection(ConstraintsIntersection(ConstraintsIntersection(), ValueSizeConstraint(0, 65535)), ValueSizeConstraint(0, 255)), ValueSizeConstraint(0, 255))))])

2.) But when I call another object 'ltmPoolMemberMonitorStatus' from a MIB that is asn1 converted MIB 'F5-BIGIP-LOCAL-MIB' it produces an error:

The only line that changes was the ObjectType line: Note: All required mibs seem to be 'loaded' after 'evaluation'

From: ObjectType(ObjectIdentity('SNMPv2-MIB', 'sysDescr'))) To: ObjectType(ObjectIdentity('F5-BIGIP-LOCAL-MIB', 'ltmPoolMemberMonitorStatus')))

#!/usr/bin/env python

from pysnmp.hlapi import *
from pprint import pprint

community = 'communitypassword'
lb = 'f5ltm-host'

g = nextCmd(SnmpEngine(),
            CommunityData(community),
            UdpTransportTarget((lb, 161)),
            ContextData(),
            ObjectType(ObjectIdentity('F5-BIGIP-LOCAL-MIB', 'ltmPoolMemberMonitorStatus')))

print next(g)

Produces:

Traceback (most recent call last): File "/home/lhoffman/bbcom/python/f5zabbix/f5walk.py", line 19, in print next(g) File "/usr/local/lib/python2.7/dist-packages/pysnmp/hlapi/asyncore/sync/cmdgen.py", line 347, in nextCmd snmpEngine.transportDispatcher.runDispatcher() File "/usr/local/lib/python2.7/dist-packages/pysnmp/carrier/asyncore/dispatch.py", line 49, in runDispatcher raise PySnmpError('poll error: %s' % ';'.join(format_exception(*exc_info()))) pysnmp.error.PySnmpError: poll error: Traceback (most recent call last): ; File "/usr/local/lib/python2.7/dist-packages/pysnmp/carrier/asyncore/dispatch.py", line 45, in runDispatcher use_poll=True, map=self.sockMap, count=1) ; File "/usr/lib/python2.7/asyncore.py", line 220, in loop poll_fun(timeout, map) ; File "/usr/lib/python2.7/asyncore.py", line 201, in poll2 readwrite(obj, flags) ; File "/usr/lib/python2.7/asyncore.py", line 123, in readwrite obj.handle_error() ; File "/usr/lib/python2.7/asyncore.py", line 108, in readwrite obj.handle_read_event() ; File "/usr/lib/python2.7/asyncore.py", line 449, in handle_read_event self.handle_read() ; File "/usr/local/lib/python2.7/dist-packages/pysnmp/carrier/asyncore/dgram/base.py", line 157, in handle_read self._cbFun(self, transportAddress, incomingMessage) ; File "/usr/local/lib/python2.7/dist-packages/pysnmp/carrier/base.py", line 68, in _cbFun self, transportDomain, transportAddress, incomingMessage ; File "/usr/local/lib/python2.7/dist-packages/pysnmp/entity/engine.py", line 145, in receiveMessageCbFun self, transportDomain, transportAddress, wholeMsg ; File "/usr/local/lib/python2.7/dist-packages/pysnmp/proto/rfc3412.py", line 458, in receiveMessage cachedParams['cbCtx']) ; File "/usr/local/lib/python2.7/dist-packages/pysnmp/entity/rfc3413/cmdgen.py", line 131, in processResponsePdu cbFun(snmpEngine, origSendRequestHandle, None, PDU, cbCtx) ; File "/usr/local/lib/python2.7/dist-packages/pysnmp/entity/rfc3413/cmdgen.py", line 274, in processResponseVarBinds varBindTable, cbCtx): ; File "/usr/local/lib/python2.7/dist-packages/pysnmp/hlapi/asyncore/cmdgen.py", line 336, in cbFun [vbProcessor.unmakeVarBinds(snmpEngine, varBindTableRow, lookupMib) for varBindTableRow in varBindTable], ; File "/usr/local/lib/python2.7/dist-packages/pysnmp/hlapi/varbinds.py", line 43, in unmakeVarBinds varBinds = [ObjectType(ObjectIdentity(x[0]), x[1]).resolveWithMib(mibViewController) for x in varBinds] ; File "/usr/local/lib/python2.7/dist-packages/pysnmp/smi/rfc1902.py", line 838, in resolveWithMib self.args[0].resolveWithMib(mibViewController) ; File "/usr/local/lib/python2.7/dist-packages/pysnmp/smi/rfc1902.py", line 446, in resolveWithMib self.indices = rowNode.getIndicesFromInstId(suffix) ; File "/tmp/pip-build-dm8da3/pysnmp/pysnmp/smi/mibs/SNMPv2-SMI.py", line 1150, in getIndicesFromInstId ; File "/tmp/pip-build-dm8da3/pysnmp/pysnmp/smi/mibs/SNMPv2-SMI.py", line 955, in setFromName ; File "/usr/local/lib/python2.7/dist-packages/pyasn1/type/univ.py", line 126, in clone return self.__class(value, tagSet, subtypeSpec, namedValues) ; File "/usr/local/lib/python2.7/dist-packages/pyasn1/type/univ.py", line 22, in init self, value, tagSet, subtypeSpec ; File "/usr/local/lib/python2.7/dist-packages/pyasn1/type/base.py", line 75, in init self._verifySubtypeSpec(value) ; File "/usr/local/lib/python2.7/dist-packages/pyasn1/type/base.py", line 33, in _verifySubtypeSpec raise c('%s at %s' % (i, self.class.name)) ;ValueConstraintError: ConstraintsIntersection(ConstraintsIntersection(ConstraintsIntersection(), ValueRangeConstraint(-2147483648, 2147483647)), ConstraintsUnion(SingleValueConstraint(0, 1, 2, 3, 4, 16))) failed at: "ConstraintsUnion(SingleValueConstraint(0, 1, 2, 3, 4, 16)) failed at: "all of (SingleValueConstraint(0, 1, 2, 3, 4, 16),) failed for "13""" at InetAddressType

Can you tell me where to find out where the 'failed for "13" is coming form?
When I run this script as 'cont' inside the python debugger (pdb -m ./f5walk.py) I see this message in the "post mortem debugging'

/usr/local/lib/python2.7/dist-packages/pysnmp/carrier/asyncore/dispatch.py(49)runDispatcher() -> raise PySnmpError('poll error: %s' % ';'.join(format_exception(*exc_info())))

etingof commented 8 years ago

You are getting the value of "13" from your SNMP agent. When pysnmp tries to map "13" to a symbolic constant for given OID (as defined by F5-BIGIP-LOCAL-MIB) it fails because "13" is not in MIB:

ltmPoolMemberMonitorStatus OBJECT-TYPE 
        SYNTAX INTEGER {
                unchecked(0),
                checking(1),
                inband(2),
                forced-up(3),
                up(4),
                addr-down(18),
                down(19),
                forced-down(20),
                maint(21),
                irule-down(22),
                inband-down(23),
                down-manual-resume(24)
        }

May be it's a matter of old MIB so consider upgrading it. Or you could add "13" manually into the MIB, however the semantics of that value would remain unknown.

leonhoffman commented 8 years ago

Thank you,

That would be strange since 13 isn't one of the values returned when doing an snmpwalk from the command line from NET-SNMP version: 5.7.3

As you can see there are only three types of values returned (4, 19, and 18)

lhoffman@keelback:~/snmpwalk -c "community-passwd" -v2c -M '+./f5/mibs-orig' -m ALL bbalb5 F5-BIGIP-LOCAL-MIB::ltmPoolMemberMonitorStatus 2>/dev/null| sed 's/.*=//' | sort | uniq -c | sort -nr
    144  INTEGER: up(4)
      8  INTEGER: down(19)
      1  INTEGER: addrDown(18)

Also this MIB file is the latest pulled from the device which has a recent (latest upgrade).

Also I am using pysnmp 4.3.2 lhoffman@keelback:/$ pip show pysnmp | grep Version Metadata-Version: 2.0 Version: 4.3.2

I'm not sure why the Agent would respond differently with these two tools. Maybe the snmpwalk tool is throwing out the '13'?

Thanks for your response.

etingof commented 8 years ago

If you simply enable pysnmp debugging:

from pysnmp import debug

debug.setLogger(debug.Debug('msgproc'))

Genuine response values will be revealed.

leonhoffman commented 8 years ago

Yeah, before I opened this issue I used this: And could not see anywhere where the 13 was coming from.

from pysnmp import debug
debug.setLogger(debug.Debug('all'))

Doing the debug line you suggested produced this as the last output;

2016-08-21 18:25:47,798 pysnmp: prepareDataElements: Message:
 version='version-2c'
 community=<removed for security>
 data=PDUs:
  response=ResponsePDU:
   request-id=4787185
   error-status='noError'
   error-index=0
   variable-bindings=VarBindList:
    VarBind:
     name=1.3.6.1.4.1.3375.2.2.5.3.2.1.11.16.47.67.111.109.109.111.110.47.97.112.105.45.101.100.103.101.13.47.67.111.109.109.111.110.47.104.111.112.115.49.80
     =_BindValue:
      value=ObjectSyntax:
       simple=SimpleSyntax:
        integer-value=4

2016-08-21 18:25:47,798 pysnmp: prepareDataElements: SM returned securityEngineId SnmpEngineID() securityName SnmpAdminString('s4221684371217059189', subtypeSpec=ConstraintsIntersection(ConstraintsIntersection(ConstraintsIntersection(ConstraintsIntersection(), ValueSizeConstraint(0, 65535)), ValueSizeConstraint(0, 255)), ValueSizeConstraint(1, 32)))
2016-08-21 18:25:47,799 pysnmp: prepareDataElements: unique PDU request-id 4787185 replaced with original ID 14117212
Traceback (most recent call last):
  File "./f5walk.py", line 21, in <module>
    print next(g)
  File "/usr/local/lib/python2.7/dist-packages/pysnmp/hlapi/asyncore/sync/cmdgen.py", line 347, in nextCmd
    snmpEngine.transportDispatcher.runDispatcher()
  File "/usr/local/lib/python2.7/dist-packages/pysnmp/carrier/asyncore/dispatch.py", line 49, in runDispatcher
    raise PySnmpError('poll error: %s' % ';'.join(format_exception(*exc_info())))
pysnmp.error.PySnmpError: poll error: Traceback (most recent call last):
;  File "/usr/local/lib/python2.7/dist-packages/pysnmp/carrier/asyncore/dispatch.py", line 45, in runDispatcher
    use_poll=True, map=self.__sockMap, count=1)
;  File "/usr/lib/python2.7/asyncore.py", line 220, in loop
    poll_fun(timeout, map)
;  File "/usr/lib/python2.7/asyncore.py", line 201, in poll2
    readwrite(obj, flags)
;  File "/usr/lib/python2.7/asyncore.py", line 123, in readwrite
    obj.handle_error()
;  File "/usr/lib/python2.7/asyncore.py", line 108, in readwrite
    obj.handle_read_event()
;  File "/usr/lib/python2.7/asyncore.py", line 449, in handle_read_event
    self.handle_read()
;  File "/usr/local/lib/python2.7/dist-packages/pysnmp/carrier/asyncore/dgram/base.py", line 157, in handle_read
    self._cbFun(self, transportAddress, incomingMessage)
;  File "/usr/local/lib/python2.7/dist-packages/pysnmp/carrier/base.py", line 68, in _cbFun
    self, transportDomain, transportAddress, incomingMessage
;  File "/usr/local/lib/python2.7/dist-packages/pysnmp/entity/engine.py", line 145, in __receiveMessageCbFun
    self, transportDomain, transportAddress, wholeMsg
;  File "/usr/local/lib/python2.7/dist-packages/pysnmp/proto/rfc3412.py", line 458, in receiveMessage
    cachedParams['cbCtx'])
;  File "/usr/local/lib/python2.7/dist-packages/pysnmp/entity/rfc3413/cmdgen.py", line 131, in processResponsePdu
    cbFun(snmpEngine, origSendRequestHandle, None, PDU, cbCtx)
;  File "/usr/local/lib/python2.7/dist-packages/pysnmp/entity/rfc3413/cmdgen.py", line 274, in processResponseVarBinds
    varBindTable, cbCtx):
;  File "/usr/local/lib/python2.7/dist-packages/pysnmp/hlapi/asyncore/cmdgen.py", line 336, in __cbFun
    [vbProcessor.unmakeVarBinds(snmpEngine, varBindTableRow, lookupMib) for varBindTableRow in varBindTable],
;  File "/usr/local/lib/python2.7/dist-packages/pysnmp/hlapi/varbinds.py", line 43, in unmakeVarBinds
    varBinds = [ObjectType(ObjectIdentity(x[0]), x[1]).resolveWithMib(mibViewController) for x in varBinds]
;  File "/usr/local/lib/python2.7/dist-packages/pysnmp/smi/rfc1902.py", line 838, in resolveWithMib
    self.__args[0].resolveWithMib(mibViewController)
;  File "/usr/local/lib/python2.7/dist-packages/pysnmp/smi/rfc1902.py", line 446, in resolveWithMib
    self.__indices = rowNode.getIndicesFromInstId(suffix)
;  File "/tmp/pip-build-dm8da3/pysnmp/pysnmp/smi/mibs/SNMPv2-SMI.py", line 1150, in getIndicesFromInstId
;  File "/tmp/pip-build-dm8da3/pysnmp/pysnmp/smi/mibs/SNMPv2-SMI.py", line 955, in setFromName
;  File "/usr/local/lib/python2.7/dist-packages/pyasn1/type/univ.py", line 126, in clone
    return self.__class__(value, tagSet, subtypeSpec, namedValues)
;  File "/usr/local/lib/python2.7/dist-packages/pyasn1/type/univ.py", line 22, in __init__
    self, value, tagSet, subtypeSpec
;  File "/usr/local/lib/python2.7/dist-packages/pyasn1/type/base.py", line 75, in __init__
    self._verifySubtypeSpec(value)
;  File "/usr/local/lib/python2.7/dist-packages/pyasn1/type/base.py", line 33, in _verifySubtypeSpec
    raise c('%s at %s' % (i, self.__class__.__name__))
;ValueConstraintError: ConstraintsIntersection(ConstraintsIntersection(ConstraintsIntersection(), ValueRangeConstraint(-2147483648, 2147483647)), SingleValueConstraint(0, 1, 2, 3, 4, 16)) failed at: "SingleValueConstraint(0, 1, 2, 3, 4, 16) failed at: "13"" at InetAddressType
leonhoffman commented 8 years ago

The InetAddressType is referring to one of the classes in the INET-ADDRESS-MIB.py which is one of the 20 mibs that are loaded when running this request; and it's true 13 isn't one of the values: but again this ASN1 mib file is the latest from the device and true 13 isn't an option. I just don't see where the agent is returning 13 when setting the debug level to 'all'

Here are the only options in that mib for InetAddressType: class InetAddressType(Integer32, TextualConvention): subtypeSpec = Integer32.subtypeSpec+SingleValueConstraint(0, 1, 2, 3, 4, 16,) namedValues = NamedValues(("unknown", 0), ("ipv4", 1), ("ipv6", 2), ("ipv4z", 3), ("ipv6z", 4), ("dns", 16),)

etingof commented 8 years ago

Ah, yes, you are right! My initial analysis was incorrect. So the problem might be in indices -- the tail part of the OID that belongs to SNMP table ltmPoolMemberTable:

ltmPoolMemberEntry OBJECT-TYPE
        SYNTAX  LtmPoolMemberEntry
        MAX-ACCESS not-accessible
        STATUS current
        DESCRIPTION
                "Columns in the ltmPoolMember Table"
        INDEX {
                ltmPoolMemberPoolName,
                ltmPoolMemberAddrType,
                ltmPoolMemberAddr,
                ltmPoolMemberPort
        }
        ::= { ltmPoolMemberTable 1 } 

So my guess is that when tail part of received OID (which is 16.47.67.111.109.109.111.110.47.97.112.105.45.101.100.103.101.13.47.67.111.109.109.111.110.47.104.111.112.115.49.80) is broken down onto the four index components as mentioned in the MIB, the part that is left for ltmPoolMemberAddr has a length of 13 octets which does not correspond to any valid IP address notation.

You could get rid of that problem by turning off MIB resolution in pysnmp, but that is not that handy. So I'm planning a more useful feature -- ignoring broken indices/values whenever they occur...

Could you further investigate the broken index hypothesis?

leonhoffman commented 8 years ago

It does look like the 13 represents the length of the string in octets. With later versions of the F5 LTM software the agent no longer returns the ip address within the OID.

And the latest mib "F5-BIGIP-LOCAL-MIB" for 'ltmPoolMemberEntry' no longer has ltmPoolMemberAddrType.

See first section below:

ltmPoolMemberEntry OBJECT-TYPE
    SYNTAX  LtmPoolMemberEntry
    MAX-ACCESS not-accessible
    STATUS current
    DESCRIPTION
        "Columns in the ltmPoolMember Table"
    INDEX {
        ltmPoolMemberPoolName,
        ltmPoolMemberNodeName,
        ltmPoolMemberPort
    }
    ::= { ltmPoolMemberTable 1 } 

But what is confusing is, after using the mibdump.py tool on this file I see this section in the .py file. As you can see the ltmPoolMemberAddrType IS listed in the ltmPoolMemberEntry. I'm not sure where it built that from but it appears incorrect to me.

ltmPoolMemberEntry = MibTableRow((1, 3, 6, 1, 4, 1, 3375, 2, 2, 5, 3, 2, 1), ).setIndexNames((0, "F5-BIGIP-LOCAL-MIB", "ltmPoolMemberPoolName"), (0, "F5-BIGIP-LOCAL-MIB", "ltmPoolMemberAddrType"), (0, "F5-BIGIP-LOCAL-MIB", "ltmPoolMemberAddr"), (0, "F5-BIGIP-LOCAL-MIB", "ltmPoolMemberPort"))

BTW: Thanks for spending time looking at this. I really appreciate it.

If you have any ideas that would be great. I will continue to try to understand where that is coming from, I think this may get me closer to find out why this is occurring. I'm sure this broken mismatch has something to do with pysnmp getting thrown off enforcing OID resolution. If I have to I may consider turning it off. Normally I don't really care, I usually care about the value. In this occasion F5 made this change (from using IP ADDR to NODENAME) and it will be handy to get the ltmPoolMemberNodeName from the resolution. I'm trying to build an low-level discovery item to send to Zabbix, and the pool node name will be needed.

leonhoffman commented 8 years ago

I found out why.

While in the path of the mibfile and passing the mibfile on the commandline for mibdump.py it still retrieves this mibfile from http://mibs.snmplabs.com/asn1/F5-BIGIP-LOCAL-MIB which is an incorrect mibfile.

python $(which mibdump.py) --debug=all F5-BIGIP-LOCAL-MIB.txt

2016-08-22 16:01:51,250 pysmi: F5-BIGIP-LOCAL-MIB read from http://mibs.snmplabs.com:80/asn1/F5-BIGIP-LOCAL-MIB and compiled by PyFileWriter{"/Users/leon/.pysnmp/mibs"}

etingof commented 8 years ago

Great! If you share the up-to-date MIBs, I could update them at mins.snmplabs.com. ;)

leonhoffman commented 8 years ago

I sure can, but after I resolved that issue I have an issue similar to the case you thought this was in the first place. When I walk the ltmPoolMemberMonitorStatus Object from the F5-BIGIP-LOCAL-MIB using the cmdGen.nextCmd it appears that in order to verify that this Object table is ended it stops when the oid is incremented. The issue after this table (ltmPoolMemberEntry 11) has ended it retrieves the first object from the next one which is (ltmPoolMemberEntry 12).

Here's the issue: this is a Deprecated Object which is why I think that the F5 is sending a "0" as all of the values. That doesn't match any of the Object Integers which are below.

ltmPoolMemberNewSessionEnable OBJECT-TYPE SYNTAX INTEGER { userDisabled(1), userEnabled(2), monitorEnabled(3), monitorDisabled(4) }

Do you know of a way to catch that Exception in a script that users use? Or are these options below users have?

1.) Fix the pysnmp module and submit a PR 2.) Hack the ASN1 mib file with a fake "0" value; which I have done for now.

leonhoffman commented 8 years ago

Here are the files mibs_f5.tar.gz

etingof commented 8 years ago

I'd really appreciate if you make pysnmp [conditionally] ignoring value conversion errors and submit PR on that!

I pushed the MIBs you sent to http://mibs.snmplabs.com/asn1/