UnboundLocalError: local variable 'index' referenced before assignment

[futureweb]

When trying to get Disk Stats with this Plugin it errors with:

# /usr/bin/python3.4 /usr/local/icinga/libexec/check-disk-snmp.py -H tmyserver.dom.tld -C public -w 80%,50%i -c 90% /
Traceback (most recent call last):
  File "/usr/local/icinga/libexec/check-disk-snmp.py", line 413, in <module>
    sys.exit(main())
  File "/usr/local/icinga/libexec/check-disk-snmp.py", line 391, in main
    raise(e)
  File "/usr/local/icinga/libexec/check-disk-snmp.py", line 367, in main
    d.lookupDiskIndex(args.mount_point)
  File "/usr/local/icinga/libexec/check-disk-snmp.py", line 220, in lookupDiskIndex
    raise(e)
  File "/usr/local/icinga/libexec/check-disk-snmp.py", line 214, in lookupDiskIndex
    if index is None:
UnboundLocalError: local variable 'index' referenced before assignment

# /usr/bin/python3.4 -V
Python 3.4.10

Already tried different mountpoints and servers, but always the same error.

Can someone point me in the right direction?

Thank you Andreas

[cinnion]

While there is a bug with the code which I am currently fixing, you should check to see if the partition shows up in the MIB data returned using snmpwalk. For example, using the following command:

snmpwalk -v2c -c public tmpserver.dom.tld iso | grep dskPath

which for one of my systems returns:

UCD-SNMP-MIB::dskPath.1 = STRING: /
UCD-SNMP-MIB::dskPath.2 = STRING: /boot
UCD-SNMP-MIB::dskPath.3 = STRING: /tmp
UCD-SNMP-MIB::dskPath.4 = STRING: /var
UCD-SNMP-MIB::dskPath.9 = STRING: /dev/shm
UCD-SNMP-MIB::dskPath.11 = STRING: /run
UCD-SNMP-MIB::dskPath.12 = STRING: /sys/fs/cgroup
UCD-SNMP-MIB::dskPath.27 = STRING: /usr
UCD-SNMP-MIB::dskPath.33 = STRING: /boot/efi
UCD-SNMP-MIB::dskPath.34 = STRING: /var/lib/mysql
UCD-SNMP-MIB::dskPath.35 = STRING: /var/lib/pgsql
UCD-SNMP-MIB::dskPath.36 = STRING: /var/lib/libvirt
UCD-SNMP-MIB::dskPath.37 = STRING: /var/lib/docker
UCD-SNMP-MIB::dskPath.39 = STRING: /run/user/0

The correct behaviour for a partition not being found should be to just output the following:

UNKNOWN Partition /var/lib/libvirt/images not found via SNMP

[cinnion]

Please try the fix-uninitialized-index branch and let me know if it fixes your problem.

[futureweb]

@cinnion - wow, that was quick! thank you! :-)

It isn't erroring out anymore but showing the right error: # /usr/bin/python3.4 /usr/local/icinga/libexec/check-disk-snmp.py -H hostname.futurehosting.at -C public -w 80%,50%i -c 90% / UNKNOWN Partition / not found via SNMP

Not quite sure why the Partitions aren't published to SMTP on our setups. (Centos/AlmaLinux 7-9) snmpwalk also returning nothing on multiple servers

[cinnion]

Andreas,

Regarding your issue of not having the partitions show up, one of my snmpd.conf files for a CentOS box has the following block in it. The same should follow with other distros like Alma and Rocky.

# disk: Check for disk space usage of a partition.
#   The agent can check the amount of available disk space, and make
#   sure it is above a set limit.  
#   
#    disk PATH [MIN=100000]
#   
#    PATH:  mount path to the disk in question.
#    MIN:   Disks with space below this value will have the Mib's errorFlag set.
#           Can be a raw integer value (units of kB) or a percentage followed by the %
#           symbol.  Default value = 100000.
#   
#   The results are reported in the dskTable section of the UCD-SNMP-MIB tree

disk    /                       20%
disk    /var                    5%
disk    /opt                    5%
disk    /homes                  10000000
disk    /backups/mssql/backups  20000000
disk    /homes/samba            18387136

includeAllDisks 10%

Take a look at the Disk Usage Monitoring section of snmpd.conf for more info.

Also note: While I have not done extensive research into it, neither opnsense nor TrueNAS appear to support the UCD-SNMP-MIB (the latter probably being due to oddities of ZFS), so in those cases, a different plugin would likely need to be written/used. And I would have to guess that pfsense would also be similar to this as well.

I will go ahead and keep this open to see if that helps you with your configuration issue for now.

[cinnion]

You will also want to pull the master branch version once I merge the fix branch. There is an additional fix included, which occurred when the SNMPD on the remote end set an error string in the dskErrorMsg OID for the partition.

[futureweb]

@cinnion - thx again - your hint regarding the snmpd.conf was right. Now when executing your check on CLI it's working:

executed using User 'icinga':

$ /usr/local/icinga/libexec/check-disk-snmp.py -H host.futurehosting.at -C public -w 10%i -c 90%i /
Warning 20 > warning threshold of 10 for dskPercentNode dskPercent=78

weirdly enough it's not working our of icinga itself - even it's running under the same user?

File /usr/lib/python3.4/site-packages/pysnmp/smi/builder.py, line 401, in loadModules
self.loadModule(modName, **userCtx)
File /usr/lib/python3.4/site-packages/pysnmp/smi/builder.py, line 379, in loadModule
modName and modName + .py[co], , .join([str(x) for x in self.__mibSources]))
pysnmp.smi.error.MibNotFoundError: MIB file UCD-SNMP-MIB.py[co] not found in search path (DirMibSource(/usr/lib/python3.4/site-packages/pysnmp/smi/mibs), DirMibSource(/usr/lib/python3.4/site-packages/pysnmp/smi/mibs/instances), DirMibSource(pysnmp_mibs), DirMibSource(/root/.pysnmp/mibs))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File /usr/local/icinga/libexec/check-disk-snmp.py, line 419, in
sys.exit(main())
File /usr/local/icinga/libexec/check-disk-snmp.py, line 397, in main
raise(e)
File /usr/local/icinga/libexec/check-disk-snmp.py, line 373, in main
d.lookupDiskIndex(args.mount_point)
File /usr/local/icinga/libexec/check-disk-snmp.py, line 222, in lookupDiskIndex
raise(e)
File /usr/local/icinga/libexec/check-disk-snmp.py, line 203, in lookupDiskIndex
for (errorIndication, errorStatus, errorIndex, varBinds) in cmd:
File /usr/lib/python3.4/site-packages/pysnmp/hlapi/asyncore/sync/cmdgen.py, line 349, in nextCmd
initialVars = [x[0] for x in vbProcessor.makeVarBinds(snmpEngine, varBinds)]
File /usr/lib/python3.4/site-packages/pysnmp/hlapi/varbinds.py, line 39, in makeVarBinds
__varBinds.append(varBind.resolveWithMib(mibViewController, ignoreErrors=False))
File /usr/lib/python3.4/site-packages/pysnmp/smi/rfc1902.py, line 853, in resolveWithMib
self.__args[0].resolveWithMib(mibViewController)
File /usr/lib/python3.4/site-packages/pysnmp/smi/rfc1902.py, line 481, in resolveWithMib
self.__modName, self.__symName
File /usr/lib/python3.4/site-packages/pysnmp/smi/builder.py, line 441, in importSymbols
self.loadModules(modName, **userCtx)
File /usr/lib/python3.4/site-packages/pysnmp/smi/builder.py, line 410, in loadModules
raise error.MibNotFoundError(%s compilation error(s): %s % (modName, errs))
pysnmp.smi.error.MibNotFoundError: UCD-SNMP-MIB compilation error(s): failure creating destination directory /root/.pysnmp/mibs: [Errno 13] Permission denied: /root/.pysnmp at MIB UCD-SNMP-MIBcaused by : MIB file UCD-SNMP-MIB.py[co] not found in search path (DirMibSource(/usr/lib/python3.4/site-packages/pysnmp/smi/mibs), DirMibSource(/usr/lib/python3.4/site-packages/pysnmp/smi/mibs/instances), DirMibSource(pysnmp_mibs), DirMibSource(/root/.pysnmp/mibs))

No idea why it's working from CLU but not from icinga itself? Maybe you have some insights into this as well? :-)

Thank you very much Andy

[futureweb]

update: had to manually copy the MIB into the right directory - still not sure why it was working fine with user icinga on CLI ...

cp /root/.pysnmp/mibs/UCD-SNMP-MIB.py /usr/lib/python3.4/site-packages/pysnmp/smi/mibs/instances/
chmod a+r /usr/lib/python3.4/site-packages/pysnmp/smi/mibs/instances/UCD-SNMP-MIB.py 

[cinnion]

Oddly, the between the pysnmp and pysmi packages, the MIB was downloaded and installed under the nagios home directory in the .pysnmp/mibs directory. I wonder if it is an issue where the user which is running the check-disk-snmp script could not write the MIB file, perhaps due to standard permissions or SELinux. Indeed, in looking at the log you sent, the process thought its home directory was /root (which makes me wonder about an SELinux block on a process which I hope is not running as root!!!). Given that it was just now that I realized that Icinga was a Nagios fork (I am going to have to check them out), perhaps the folks in that community might be able to answer it better than I (but you might also check /var/log/audit.log for denials as well). Be sure to include whether or not your system has SELinux in a permissive or enforcing setting. You can use the getenforce or sestatus commands for that information.

But I am glad you got it working, and I am going to go ahead and merge the branch to master and close this issue.