Support for NetBox 3.5.4?

bitcollector1 commented 1 year ago

Support for NetBox 3.5.4? I was getting an error when upgrading NetBox today to 3.5.4

It failed 2x, until I removed this from PLUGINS in the configuration.py for NetBox.

Applying database migrations (python3 netbox/manage.py migrate)...
Traceback (most recent call last):
  File "netbox/manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line
    utility.execute()
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/core/management/__init__.py", line 440, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/core/management/base.py", line 402, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/core/management/base.py", line 448, in execute
    output = self.handle(*args, **options)
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/core/management/base.py", line 96, in wrapped
    res = handle_func(*args, **kwargs)
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/core/management/commands/migrate.py", line 97, in handle
    self.check(databases=[database])
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/core/management/base.py", line 475, in check
    all_issues = checks.run_checks(
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/core/checks/registry.py", line 88, in run_checks
    new_errors = check(app_configs=app_configs, databases=databases)
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/core/checks/urls.py", line 14, in check_url_config
    return check_resolver(resolver)
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/core/checks/urls.py", line 24, in check_resolver
    return check_method()
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/urls/resolvers.py", line 494, in check
    for pattern in self.url_patterns:
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/utils/functional.py", line 57, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/urls/resolvers.py", line 715, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/utils/functional.py", line 57, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/urls/resolvers.py", line 708, in urlconf_module
    return import_module(self.urlconf_name)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/ebay/home/ebay/netbox/netbox/netbox/urls.py", line 8, in <module>
    from extras.plugins.urls import plugin_admin_patterns, plugin_patterns, plugin_api_patterns
  File "/ebay/home/ebay/netbox/netbox/extras/plugins/urls.py", line 31, in <module>
    urlpatterns = import_string(f"{plugin_path}.urls.urlpatterns")
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/utils/module_loading.py", line 30, in import_string
    return cached_import(module_path, class_name)
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/django/utils/module_loading.py", line 15, in cached_import
    module = import_module(module_path)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/netbox_kea/urls.py", line 4, in <module>
    from . import views
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/netbox_kea/views.py", line 17, in <module>
    from . import constants, forms, tables
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/netbox_kea/forms.py", line 12, in <module>
    from .utilities import is_hex_string
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/netbox_kea/utilities.py", line 83, in <module>
    class OptionalViewTab(ViewTab):
  File "/ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/netbox_kea/utilities.py", line 84, in OptionalViewTab
    def __init__(self, *args, is_enabled: Callable[[Any], bool], **kwargs) -> None:
TypeError: 'ABCMeta' object is not subscriptable

devon-mar commented 1 year ago

Did it work on NetBox v3.5.3 for you?

Could you try making the following changes to /ebay/home/ebay/netbox/venv/lib/python3.8/site-packages/netbox_kea/utilities.py:

-from collections.abc import Callable
 from datetime import datetime
-from typing import Any, Dict, List, Literal, Optional
+from typing import Any, Callable, Dict, List, Literal, Optional

 from django.http import HttpResponse
 from django.shortcuts import redirect

bitcollector1 commented 1 year ago

Your suggestion worked perfectly and so is the plugin! Nice work and thanks for sharing!

bitcollector1 commented 1 year ago

one other thing, I think your command is not correct and should be

./manage.py migrate

instead of

./migrate.py migrate

bitcollector1 commented 1 year ago

would it also be possible to report on pool info and utilization for a given subnet?

server.dhcp4.statistic_get('subnet[1019156019].total-addresses')

KeaResponse(result=0, text=None, arguments={'subnet[1019156019].total-addresses': [[29, '2023-06-07 12:24:17.224493']]})

server.dhcp4.statistic_get('subnet[1019156019].assigned-addresses')

KeaResponse(result=0, text=None, arguments={'subnet[1019156019].assigned-addresses': [[12, '2023-06-07 12:47:29.204784'], [11, '2023-06-07 12:35:42.945926'], [10, '2023-06-07 12:34:32.895585'], [9, '2023-06-07 12:34:31.810025'], [8, '2023-06-07 12:34:30.858006'], [7, '2023-06-07 12:34:30.410463'], [6, '2023-06-07 12:34:08.498306'], [5, '2023-06-07 12:34:08.120871'], [4, '2023-06-07 12:34:03.850084'], [3, '2023-06-07 12:32:41.549737'], [2, '2023-06-07 12:32:39.650907'], [1, '2023-06-07 12:32:15.823833'], [0, '2023-06-07 12:24:17.224847']]})

bitcollector1 commented 1 year ago

pool info seems to be available with this call

test = server.dhcp4.cached_config
print(test['Dhcp4']['shared-networks'][0]['subnet4'][0]['pools'][0]['pool'])

10.254.198.129-10.254.198.177

devon-mar commented 1 year ago

Your suggestion worked perfectly and so is the plugin! Nice work and thanks for sharing!

Great! I've published v0.1.1 with the change so you can update to that.

one other thing, I think your command is not correct and should be

Fixed.

would it also be possible to report on pool info and utilization for a given subnet?

I think this is a good use case for Stork. I'm hesitant to implement this feature because Kea doesn't have a proper API to get pool/subnet information. Currently this plugin just uses config-get to get the list of subnets which may be expensive, (I haven't actually confirmed this myself). None the less, getting the configuration just to get the list of configured subnets/pools doesn't seem like a very good approach.

bitcollector1 commented 1 year ago

it appears that our company paid for the premium API support so we can make the direct API calls to pull subnet info

network = server.dhcp4.network4_get('sjc-lab-33-ztp-data')

print(network.name) 
print(network.subnet4[0].subnet)
print(network.subnet4[0].pools[0].pool)

sjc-lab-33-ztp-data
10.254.142.128/26
10.254.142.160-10.254.142.190

bitcollector1 commented 1 year ago

FYI, I'm using this new wrapper https://github.com/BSpendlove/pykeadhcp/blob/main/COMMANDS.md

bitcollector1 commented 1 year ago

One last question, I'm not seeing anything for DHCP Subnets.....curious if I'm missing something basic as I'm new to KEA.

The DHCP Leases is working great.

bitcollector1 commented 1 year ago

Here you can see the DHCP info is available via the python calls.

subnets_v4 = server.dhcp4.subnet4_list()

for i in subnets_v4:
    print(str(i.id).ljust(12), i.subnet )

416769625    10.254.85.128/25
440383492    10.254.141.0/26
593730608    10.254.151.48/28
602504513    10.254.86.0/25
605481152    10.254.151.32/28
830827846    10.254.142.192/26
1019156019   10.254.142.96/27
1818034871   10.254.148.128/25
1986588591   10.254.199.0/27
2086002595   10.254.142.128/26
2149123380   10.254.150.128/26
2192260479   10.254.141.64/26
2419609915   10.254.199.64/26
2444456677   10.254.142.64/27
2494219477   10.254.84.0/25
2541440121   10.254.148.0/25
2554840638   10.254.84.128/25
2910585232   10.254.143.0/26
2940872388   10.254.198.128/26
3013529256   10.254.150.0/26
3035992152   10.254.141.192/26
3179841559   10.254.142.32/27
3185248078   10.254.199.32/27
3436875955   10.254.149.0/25
3452549073   10.254.150.192/26
3473610783   10.254.85.0/25
3591900418   10.254.151.16/28
3784171495   10.254.151.160/27
3794378056   10.254.151.0/28
3801781281   10.254.198.192/26
3972899536   10.254.149.128/25
4030412670   10.254.150.64/26
4144329181   10.254.142.0/27
4179960509   10.254.141.128/26

bitcollector1 commented 1 year ago

Stork looks pretty cool but might be overkill for my needs, I already have so many tools setup :)

devon-mar commented 1 year ago

it appears that our company paid for the premium API support so we can make the direct API calls to pull subnet info.

I don't have access to that hook library so this plugin just pulls the list of subnets from the config.

One last question, I'm not seeing anything for DHCP Subnets.....curious if I'm missing something basic as I'm new to KEA.

Are you using a database to store your subnets? If so, this plugin grabs the lists of subnets from config-get so the subnets might not be in the configuration. I've only tested this plugin with subnets configured in the JSON config file.

If you run config-get do you see the subnets?

Here you can see the DHCP info is available via the python calls.

It looks like this command uses the subnet4-list command (which is from the hook library).

Stork looks pretty cool but might be overkill for my needs, I already have so many tools setup :)

Yeah it does have quite a bit of functionality, mainly around monitoring.

bitcollector1 commented 1 year ago

KEA was setup using the databases for config, host and lease information AFAIK, I'm still learning this on the fly and did not set it all up myself, but we don't seem to have a configuration file we can edit like we did with the old DHCP server.

That said, The subnets do show in the configuration, but they were configured in "shared-networks".

'shared-networks': [{'name': 'sjc-ctlab-03-r208-mgmt1',
    'option-data': [],
    'relay': {'ip-addresses': []},
    'subnet4': [{'4o6-interface': '',
      '4o6-interface-id': '',
      '4o6-subnet': '',
      'id': 2940872388,
      'next-server': '10.254.141.17',
      'option-data': [{'always-send': True,
        'code': 28,
        'csv-format': True,
        'data': '10.254.198.191',
        'name': 'broadcast-address',
        'never-send': False,
        'space': 'dhcp4'},
       {'always-send': True,
        'code': 1,
        'csv-format': True,
        'data': '255.255.255.192',
        'name': 'subnet-mask',
        'never-send': False,
        'space': 'dhcp4'},
       {'always-send': True,
        'code': 3,
        'csv-format': True,
        'data': '10.254.198.129',
        'name': 'routers',
        'never-send': False,
        'space': 'dhcp4'},
       {'always-send': True,
        'code': 15,
        'csv-format': True,
        'data': 'hwe.ebay.com',
        'name': 'domain-name',
        'never-send': False,
        'space': 'dhcp4'},
       {'always-send': True,
        'code': 119,
        'csv-format': True,
        'data': 'hwe.ebay.com.,vip.ebay.com.,ebay.com.',
        'name': 'domain-search',
        'never-send': False,
        'space': 'dhcp4'}],
      'pools': [{'option-data': [], 'pool': '10.254.198.129-10.254.198.177'}],
      'relay': {'ip-addresses': []},
      'reservations': [],
      'subnet': '10.254.198.128/26'}]},

I might have to setup stork, because who does not love more monitoring and cool grafana dashboards :)

Thanks for the tips and for developing this plugin, it's much appreciated.

bitcollector1 commented 1 year ago

I wonder how hard it would be to hack at your plugin to make the subnets and ID's show up for me since I have access to the premium hooks. It looks like it's a super simpler API call to grab this data and print it in NetBox.

This might be a nice way to ease into the plugin universe :)

devon-mar commented 1 year ago

The subnets do show in the configuration, but they were configured in "shared-networks".

I'll see if I can add support for that.

I wonder how hard it would be to hack at your plugin to make the subnets and ID's show up for me since I have access to the premium hooks.

I think you just need to modify one method.

bitcollector1 commented 1 year ago

Thanks again for creating this plugin, it's working really well and much appreciated!

devon-mar / netbox-kea

Support for NetBox 3.5.4? #9