Add Device Type Components - Module Bays and Console Ports

[kennethdammyr]

Hi

When updating a device with the _script add_device_typecomponents.py, both interfaces, module bays and console ports are populated. However, the tab menues for module bays and console ports does not get updated with the new data. In this screenshot, you may see how the device has got 17 module bays, but only showing 1 (which was there from before) in the menu:

If there are no module bays or console ports, the tab menu item is not displayed. Hence you will not have the oportunity to use them. As shown here:

Interfaces works as expected. Any idea what could cause this?

Netbox v3.6.6

[BriacD]

I just noticed this issue with PDUs and adding Power Outlets. Have you come across a solution? Script works great and does exactly what I've been needing but this is making it hard to move forward with all the changes I need to make.

[UniverseSandbox]

Same Issue here. Does someone have a solution?

[BriacD]

I upgraded to the latest version and that fixed my issue.

[ip-rx]

This is also affecting me. v3.7.4

Same issue. Ran script and created missing components (Interfaces, Console Port, PSU Module bays). Only Interfaces are accessible directly via the device's page. To find Console or PSU Mod Bays, I have to search them up in components list.

Not breaking, but certainly an annoyance.

With console port from script only

Added a dummy console port

[ip-rx]

@candlerb found a fix and already submitted a PR for it

https://github.com/netbox-community/customizations/pull/101

my team has tested this and it works.

[ip-rx]

The next issue with this bug:

Any devices that were created and affected by this bug, will still have an incorrect count of objects.

device.consoleports.count() (actual) and device.console_port_count (meta) will be inconsistent. The fix to the script calls a .save() which engages the function utilities.counters.update_counter(), but the bug caused device.console_port_count to be permanently wrong.

I threw together a script that just calcs the delta and feeds the inputs into utilities.counters.update_counter(). You can run it against any device pretty safely as it gets the literal count of each component and feeds the correct delta (positive or negative integer) into update_counter()

This logic could easily be inserted into the main script as a way to fix the issue retroactively when ran on a device.

"""
This script runs a counter check on each device's components and updates discrepancies
"""

from dcim.models import Manufacturer, DeviceType, Device
from extras.scripts import Script, ObjectVar, MultiObjectVar
from utilities.counters import update_counter

class FixDeviceComponentCounts(Script):
    class Meta:
        name = "Fix Device Component Counts"
        description = (
            "run a counter check on each device's components and updates discrepancies"
        )

    manufacturer = ObjectVar(
        model=Manufacturer,
        required=False,
    )
    device_type = ObjectVar(
        model=DeviceType,
        query_params={
            "manufacturer_id": "$manufacturer",
        },
        required=False,
    )
    devices = MultiObjectVar(
        model=Device,
        query_params={
            "device_type_id": "$device_type",
        },
    )

    def run(self, data, commit):
        for device in data["devices"]:
            device.full_clean()
            for counter_name, count_attr_name in [
                ("console_port_count", "consoleports"),
                ("console_server_port_count", "consoleserverports"),
                ("power_port_count", "powerports"),
                ("power_outlet_count", "poweroutlets"),
                ("interfaces_count", "interfaces"),
                ("rear_port_count", "rearports"),
                ("front_port_count", "frontports"),
                ("device_bay_count", "devicebays"),
                ("module_bay_count", "modulebays"),
            ]:
                actual_attr_count = getattr(device, count_attr_name).count()
                expected_attr_count = getattr(device, counter_name)
                # delta = device.consoleports.count() - device.console_port_count
                delta = actual_attr_count - expected_attr_count
                if delta != 0:
                    update_counter(device._meta.model, device.pk, counter_name, delta)

                    self.log_success(
                        "%s (%d): updated %s from %d to %d"
                        % (
                            device.name,
                            device.id,
                            counter_name,
                            expected_attr_count,
                            actual_attr_count,
                        )
                    )

Device with incorrect counts

Execution of script above

Same device with corrected counts

[kkthxbye-code]

@ip-rx - there's a management command for that.

https://github.com/netbox-community/netbox/blob/develop/netbox/utilities/management/commands/calculate_cached_counts.py

[candlerb]

That's the right answer, or more succinctly:

/opt/netbox/venv/bin/python /opt/netbox/netbox/manage.py calculate_cached_counts

will fix the problem.

[ip-rx]

Even better, thanks!

[jburgon]

When I run this script I get an error about the module bays. Anyone know what this might mean? If I comment out the line for module bays then it works. Obviously module bays don't get applied but everything else does.

`An exception occurred: IntegrityError: null value in column "level" of relation "dcim_modulebay" violates not-null constraint DETAIL: Failing row contains (2024-09-20 18:40:03.531782+00, 2024-09-20 18:40:03.531805+00, {}, 261, PIM1, PIM00000001, Mini-PIM 1, PIM1, , 118, null, null, null, null, null, null).

Traceback (most recent call last): File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute return self.cursor.execute(sql, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/netbox/venv/lib64/python3.11/site-packages/psycopg/cursor.py", line 97, in execute raise ex.with_traceback(None) psycopg.errors.NotNullViolation: null value in column "level" of relation "dcim_modulebay" violates not-null constraint DETAIL: Failing row contains (2024-09-20 18:40:03.531782+00, 2024-09-20 18:40:03.531805+00, {}, 261, PIM1, PIM00000001, Mini-PIM 1, PIM1, , 118, null, null, null, null, null, null).

The above exception was the direct cause of the following exception:

Traceback (most recent call last): File "/opt/netbox/netbox/extras/jobs.py", line 45, in run_script script.output = script.run(data, commit) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/netbox/netbox/scripts/add_device_type_components.py", line 66, in run klass.objects.bulk_create(items) File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/models/query.py", line 835, in bulk_create returned_columns = self._batched_insert( ^^^^^^^^^^^^^^^^^^^^^ File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/models/query.py", line 1875, in _batched_insert self._insert( File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/models/query.py", line 1847, in _insert return query.get_compiler(using=using).execute_sql(returning_fields) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/models/sql/compiler.py", line 1823, in execute_sql cursor.execute(sql, params) File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/backends/utils.py", line 79, in execute return self._execute_with_wrappers( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers return executor(sql, params, many, context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/backends/utils.py", line 100, in _execute with self.db.wrap_database_errors: File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/utils.py", line 91, in exit raise dj_exc_value.with_traceback(traceback) from exc_value File "/opt/netbox/venv/lib64/python3.11/site-packages/django/db/backends/utils.py", line 105, in _execute return self.cursor.execute(sql, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/opt/netbox/venv/lib64/python3.11/site-packages/psycopg/cursor.py", line 97, in execute raise ex.with_traceback(None) django.db.utils.IntegrityError: null value in column "level" of relation "dcim_modulebay" violates not-null constraint DETAIL: Failing row contains (2024-09-20 18:40:03.531782+00, 2024-09-20 18:40:03.531805+00, {}, 261, PIM1, PIM00000001, Mini-PIM 1, PIM1, , 118, null, null, null, null, null, null).`

[candlerb]

I guess it's because the module model changed in Netbox v4.1.0 to allow nested (hierarchical) modules.