search

etingof / pysnmp

Python SNMP library
http://snmplabs.com/pysnmp/
BSD 2-Clause "Simplified" License
582 stars 200 forks source link

Asyncio code not returning multiple results for nextCmd #231

Closed ajwillo closed 5 years ago

ajwillo commented 5 years ago

Good Evening,

I am migrating a script to be asynchronous and am using the asyncio library with pysnmp and what looks to be almost identical code between the synchronous and non synchronous script it pulling a lot of data for the sync code but what seems like only 1 entry for the async code.

using pysnmp==4.4.8

async walk:

    @asyncio.coroutine
    def snmp_walk(ip, oid, snmp_user, snmp_auth, snmp_priv):
        results=[]
        snmpEngine = SnmpEngine()
        errorIndication, errorStatus, errorIndex, varBinds = yield from nextCmd(
                                snmpEngine,
                                UsmUserData(snmp_user, snmp_auth, snmp_priv,
                                authProtocol=usmHMACSHAAuthProtocol,
                                privProtocol=usmAesCfb128Protocol),
                                UdpTransportTarget((ip, 161)),
                                ContextData(),
                                ObjectType(ObjectIdentity(oid)),
                                lexicographicMode=False
                            )
        if errorIndication:
            print(errorIndication)
        elif errorStatus:
            print('%s at %s' % (
                errorStatus.prettyPrint(),
                errorIndex and varBinds[int(errorIndex) - 1][0] or '?'
            )
                  )
        else:
            for varBind in varBinds:
                results.append(' = '.join([x.prettyPrint() for x in varBind]))
        snmpEngine.transportDispatcher.closeDispatcher()
        print(results)
        return results

sync walk:

from pysnmp.hlapi import *
    def snmp_walk(ip, oid, snmp_user, snmp_auth, snmp_priv):
        results=[]
        for (errorIndication,
            errorStatus,
            errorIndex,
            varBinds) in nextCmd(SnmpEngine(),
                                UsmUserData(snmp_user, snmp_auth, snmp_priv,
                                authProtocol=usmHMACSHAAuthProtocol,
                                privProtocol=usmAesCfb128Protocol),
                                UdpTransportTarget((ip, 161)),
                                ContextData(),
                                ObjectType(ObjectIdentity(oid)),
                                lexicographicMode=False):
            if errorIndication:
                print(errorIndication)
                break
            elif errorStatus:
                print('%s at %s' % (errorStatus.prettyPrint(),
                                    errorIndex and varBinds[int(errorIndex) - 1][0] or '?'))
                break
            else:
                for varBind in varBinds:
                    results.append(' = '.join([x.prettyPrint() for x in varBind]))
        return results

they both get passed the same info OIDs and creds

Sync results:

['SNMPv2-SMI::mib-2.4.24.4.1.4.0.0.0.0.0.0.0.0.0.10.10.10.2 = 10.10.10.2', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.55.1.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.11.1.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.55.3.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.11.2.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.55.5.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.11.3.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.55.7.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.11.4.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1', 'SNMPv2-SMI::mib-2.4.24.4.1.4.10.55.9.0.255.255.255.0.0.10.10.10.1 = 10.10.10.1'...

async results:

['SNMPv2-SMI::mib-2.4.24.4.1.4.0.0.0.0.0.0.0.0.0.10.10.10.2 = 10.10.10.2']

which looks like the nextCmd isn't working possibly?

etingof commented 5 years ago

It seems that your async script misses a loop over nextCmd() while the sync version has a for-loop. Here is the example.

ajwillo commented 5 years ago

im not quite sure where I'm missing the loop?

ive tried what I thought was right below but it prints the first record many times instead of the next record

def snmp_walk_a(ip, oid, snmp_user, snmp_auth, snmp_priv):
    snmpEngine = SnmpEngine()
    while True:
        (errorIndication,
         errorStatus,
         errorIndex,
         varBindTable) = yield from nextCmd(
                            snmpEngine,
                            UsmUserData(snmp_user, snmp_auth, snmp_priv,
                            authProtocol=usmHMACSHAAuthProtocol,
                            privProtocol=usmAesCfb128Protocol),
                            UdpTransportTarget((ip, 161)),
                            ContextData(),
                            ObjectType(ObjectIdentity(oid)),
                            lexicographicMode=False
                        )

        if errorIndication:
            print(errorIndication)
            break
        elif errorStatus:
            print('%s at %s' % (
                errorStatus.prettyPrint(),
                errorIndex and varBinds[int(errorIndex) - 1][0] or '?'
            )
                  )
        else:
            for varBindRow in varBindTable:
                for varBind in varBindRow:
                    print(' = '.join([x.prettyPrint() for x in varBind]))

        varBinds = varBindTable[-1]
        if isEndOfMib(varBinds):
            break
ajwillo commented 5 years ago

Would anyone be able to assist me with this? is it a bug or an error in my code?

Thanks

etingof commented 5 years ago

This is still not quite clear what's wrong in your situation. Look, here is practically your script which works alright against public SNMP agent (you can just run this script locally):


import asyncio
from pysnmp.hlapi.v3arch.asyncio import *

@asyncio.coroutine
def run(varBinds):
    snmpEngine = SnmpEngine()
    while True:
        (errorIndication,
         errorStatus,
         errorIndex,
         varBindTable) = yield from nextCmd(
                            snmpEngine,
                            UsmUserData('usr-none-none'),
                            UdpTransportTarget(('demo.snmplabs.com', 161)),
                            ContextData(),
                            *varBinds,
                            lexicographicMode=False
                        )

        if errorIndication:
            print(errorIndication)
            break
        elif errorStatus:
            print('%s at %s' % (
                errorStatus.prettyPrint(),
                errorIndex and varBinds[int(errorIndex) - 1][0] or '?'
            )
                  )
        else:
            for varBindRow in varBindTable:
                for varBind in varBindRow:
                    print(' = '.join([x.prettyPrint() for x in varBind]))

        varBinds = varBindTable[-1]
        if isEndOfMib(varBinds):
            break

    snmpEngine.transportDispatcher.closeDispatcher()

loop = asyncio.get_event_loop()
loop.run_until_complete(
    run([ObjectType(ObjectIdentity('SNMPv2-MIB', 'sysDescr'))])
)

May be it has something to do with the rest of the code? Or with the agent you are querying?

I do not see anything suspicious in the function you've pasted.

WDYT?

ajwillo commented 5 years ago

ok, I copy and pasted your code exactly and then added my variables into create the below:

@asyncio.coroutine
def snmp_walk(ip, varBinds, snmp_user, snmp_auth, snmp_priv):
    results=[]
    print(ip)
    snmpEngine = SnmpEngine()
    i = 0
    while True:
        i = i+1
        print('Routes Found {}'.format(i))
        (errorIndication, errorStatus, errorIndex, varBindTable) = yield from nextCmd(
                            snmpEngine,
                            UsmUserData(
                                    snmp_user, 
                                    snmp_auth,
                                    snmp_priv,
                                    authProtocol=usmHMACSHAAuthProtocol,
                                    privProtocol=usmAesCfb128Protocol
                                ),
                            UdpTransportTarget((ip, 161)),
                            ContextData(),
                            *varBinds,
                            lexicographicMode=False
                        )

        if errorIndication:
            print(errorIndication)
            break
        elif errorStatus:
            print('%s at %s' % (
                errorStatus.prettyPrint(),
                errorIndex and varBinds[int(errorIndex) - 1][0] or '?'
            )
                  )
        else:
            for varBindRow in varBindTable:
                for varBind in varBindRow:
                    results.append(' = '.join([x.prettyPrint() for x in varBind]))

        varBinds = varBindTable[-1]
        if isEndOfMib(varBinds):
            break

    snmpEngine.transportDispatcher.closeDispatcher()
    return results

and its called with this

routing_table = await snmp_walk(device.poll_ip, [ObjectType(ObjectIdentity(device.routing_table_oid))], device.device.snmp_data.name, \
                                            device.device.snmp_data.auth, device.device.snmp_data.priv)

however I think I may have some error in there still, as there are 600ish routes in my table, however the script went crazy and was printing out "Routes Found 4483" before I quit it manually.

perhaps the while loop isn't breaking? are you able to advise where my issue may be?

Thanks

etingof commented 5 years ago

Ah, now the thing is that async API does not implement the lexicographicMode option. You may want to check out issue #233. In essence, here is how to break out of the SNMP GETNEXT loop:

@asyncio.coroutine
def run(varBinds):
    snmpEngine = SnmpEngine()
    initialVarBinds = varBinds
    while True:
        (errorIndication,
         errorStatus,
         errorIndex,
         varBindTable) = yield from nextCmd(
            snmpEngine,
            CommunityData('public'),
            UdpTransportTarget(('demo.snmplabs.com', 161)),
            ContextData(),
            *varBinds)

        if errorIndication:
            print(errorIndication)
            break
        elif errorStatus:
            print('%s at %s' % (
                errorStatus.prettyPrint(),
                errorIndex and varBinds[int(errorIndex) - 1][0] or '?'
            )
                  )
        else:
            for varBindRow in varBindTable:
                for idx, varBind in enumerate(varBindRow):
                    print(' = '.join([x.prettyPrint() for x in varBind]))

        varBinds = varBindTable[-1]
        if isEndOfMib(varBinds):
            break

        for varBind in varBinds:
            if initialVarBinds[0][idx].isPrefixOf(varBind[0]):
                break

        else:
            break
ajwillo commented 5 years ago

Thankyou for your help with this one, all is now working as desired