Expand InventoryItem uniqueness constraint to include assigned component

[kkthxbye-code]

NetBox version

v3.6.1

Feature type

Data model extension

Proposed functionality

Allow InventoryItems assigned to different components to have the same name.

Use case

With the change in https://github.com/netbox-community/netbox/issues/13741 adding SFP's as inventory items require a unique name, even if the inventory item is assigned to different objects (interfaces). As such it will require all SFP's, even if identical to have an incrementing integer or similar appended. This is unnecessary as the component association already highlight what interface the item belongs to.

Database changes

Extend the constraint to include (device, component, name).

External dependencies

None

See https://github.com/netbox-community/netbox/issues/13741 for discussion. If this issue is not accepted before v3.6.2 it can be closed, as I don't believe we can cleanly modify the migration added in https://github.com/netbox-community/netbox/pull/13786 retroactively.

[jeremystretch]

If this issue is not accepted before v3.6.2 it can be closed

Or we can temporarily revert #13786 to allow sufficient time for discussion. None of this was written in stone.

[jeremystretch]

@aaron-iles can you share your thoughts here as the reporter of #13741?

[PieterL75]

We populate the Inventory items only with the specific hardware used (like "Intel Xeon E5-2660 v3 2.60GHz 10 cores"). The 'Name' field contains the hardware, the 'label' field contains the 'Memory','CPU','fan',... If the uniqueness constrain is in place, then we need to add some value to the 'name' which makes it incorrect.

As this constrain was not in place since the start, the 'bugfix' really feels like a 'new feature'.

It would be appreciated to remove the constrain on uniqueness for inventory items.

[jeremystretch]

Related: #13776

[jeremystretch]

In order of increasing liberty, it seems the competing options are:

1. Require a unique name per parent (if any) and device (#13741).
2. Require a unique name per parent (if any), component (if any), and device (this FR).
3. Do not require item names to be unique.
4. Do not require item names at all (#13776).

[aaron-iles]

@aaron-iles can you share your thoughts here as the reporter of #13741?

I understand the concerns that the others are voicing here. The issue I see is interacting with NetBox via the API like the Ansible collection does. Take this task as an example:

-  name: Update some custom field of an inventory item
  netbox.netbox.netbox_inventory_item:
    netbox_ur: https://example.com
    netbox_token: 1234567890
    data:
      name: CPU 0
      device: server1.com
      custom_fields:
        cpu_speed: 4.1 GHz

If there are duplicate inventory items associated with the device server1.com, the result of this task will be unpredictable. Which inventory item named "CPU 0" would be found and updated?

The name field is always used as the unique identifier in NetBox for any object within a certain scope (ie: device names are unique per site). Having non-unique names for inventory items with a parent device (and no parent inventory item) seems to be the exception to the way NetBox normally works.

For context, we use the name reported by the server's out-of-band management device to determine the name of the inventory item. (ie: "PCIe SSD Slot 0 Bay1", "NIC.Slot.4"...)

[josh-d-smith]

Initial Thoughts

I’m having a hard time understanding why Inventory items should have component included in the unique constraint. Scrolling through the models and searching for UniqueConstraint showcases that most of the models use name as part of the unique constraint. This behavior of requiring a name and using it as part of the unique constraint seems to be usual/normal behavior.

Adding component to this mix now adds more complexity and more required fields to use when searching/filtering via the API.

Duplicate Items

Addressing the use of SFPs as inventory items, how would you uniquely identify the inventory items? I understand mapping the SFP component to an interface. If I were tracking SFPs, I would be using the slot/port name/number as part of the name to make searches easier to locate each SFP and to add uniqueness.

Example (some combination of location + item)

• ETH1/1_SFP
• ETH2/1_SFP
• ETH3/1_SFP

The subject of duplicate fans also came up such as multiple fans in a server. Let’s take a Dell PowerEdge 450 for example. These have 7 fans total that are needed to properly cool the machine. The iDrac (lights out management) will report 7 unique fans with the following names. These names also correspond to some combination of location + item.

• FAN 1
• FAN 2
• FAN 3
• FAN 4
• FAN 5
• FAN 6
• FAN 7

The subject of tracking duplicate CPUs in the same server also came up. On the operating system, the CPUs will be identified by number (starting at 0 or 1 depending on the OS and count up from there). Lights out management systems will also count sockets via number. This means that both the operating system and the lights out management system usually address the names via some combination of location + item. I would think that the exact model number should go into a field such as PartID, Description, or some other custom field that’s desired instead of the name.

• CPU 0
• CPU 1
• CPU 2
• CPU 3

Summary

I think the use case of using the same name across inventory items with the same parent or lack of parent is against the normal use case. Most situations allow for a combination of location + item to be used as the name which is unique. I understand the concerns for using duplicate inventory items (Opticals, Fans, CPUs), but I don’t think it’s ideal to use the same name for each item.

For my use cases, I’ve never ran into the situation where I needed a duplicate name for an inventory item that’s at the same parent level (including a null parent). Any case where I’ve come close to needing a duplicate name, I’ve always used some combination of location + name. Duplicate names in inventory items at the same parent level cause search/filtering issues as @aaron-iles has pointed out above.

I vote to require a unique name per parent (if any) or device as per #13741

[kkthxbye-code]

@josh-d-smith

Adding component to this mix now adds more complexity and more required fields

You seem confused as to what is proposed in this FR. It is proposed to add the component in the same way parent is part of the unique constraint which would make it consistent as parent and component pretty much achieves the same thing, just for different object types. Your comment seem under the impression that component would be a required field.

Adding component to this mix now adds more complexity

Could you elaborate on how adding constraints add complexity? If more constraints = more complexity, would it be better to remove them all? Others have suggested it and my use case works fine that way.

I vote to require a unique name per parent (if any) or device

So this FR suggests: Require unique name per parent (if any), per component (if any) or device.

Addressing the use of SFPs as inventory items, how would you uniquely identify the inventory items?

There's an abundance of ways, but most common would be per serial or from what component it is assigned to. What you are suggesting is adding duplicate information in a plaintext field (the port number included in the name), when that is already specified in an actual database relation. This is just a bad suggestion and your only use case provided is "easy searching". Not sure why anyone would search for port name when looking for a SFP instead of going to the interface where the SFP is located or searching for the serial, this seems like a case of having incomplete data of ones network. Netbox also explicitly allow moving inventory items, in which case the name field (which you suggest should actually hold both name and location) will be wrong.

Also it would be helpful if you could document why the uniqueness constraint you want should apply to all users of netbox, as there are clearly other use cases than your own you want to block. As the use case of wanting unique names is easily achievable with custom validators, when the use case of wanting lax name rules cannot be.

Lastly the searching argument falls completely flat as duplicate names are still allowed on the same device with the PR, when the parent item is different. The global search doesn't even show the device either, so if using that to search you'll get duplicate named inventory items from different devices.

[kkthxbye-code]

Duplicate names in inventory items at the same parent level cause search/filtering issues as @aaron-iles has pointed out above.

The issue I see is interacting with NetBox via the API like the Ansible collection does. Take this task as an example:

Could you guys demonstrate this with the API and not ansible. It's hard to get a gauge on what your believe to be the problem, as the ansible code must does two requests. One to get the item and one to update it. How would the ansible snippet you posted deal with duplicate named inventory items with differing parents @aaron-iles?

The name field is always used as the unique identifier in NetBox for any object within a certain scope

I don't believe this to be correct, the most obvious example being devices not requiring a name.

[aaron-iles]

Could you guys demonstrate this with the API and not ansible.

Sure thing. The equivalent GET to the API from my Ansible example above is the following:

curl -X GET "https://example.com/dcim/inventory-items/?name=CPU+0&device_id=99"

If there were multiple inventory items named "CPU 0" under device 99, this would return multiple results. Ansible (in the example I wrote above) will err saying "More than one result returned for CPU 0".

[kkthxbye-code]

That's not the issue though is it? That will still be the case even if the reverted PR is reintroduced as duplicate names can still be present when the parent is different. If that really is the problem and the ansible collection solely allows filtering on name + device id, you should raise an issue in the ansible collection repo, as that is the actual problem.

Here's how you would filter a SFP by serial, absolutely no issue:

http://127.0.0.1:8000/api/dcim/inventory-items/?name=SFP&serial=myserial&device_id=117

I think it would also be relevant to know how you and @josh-d-smith get in the state of having multiple identical inventory items on the same device (where all filterable fields are identical). Is this faulty automation or user error? Either case can be trivially solved with custom validators btw.

[jeremystretch]

Here's how you would filter a SFP by serial, absolutely no issue

Filtering by serial number is feasible only in cases where you know the value beforehand, unlike a name, which can generally be presumed programmatically (e.g. "CPU 0" in the example above). It also requires the complete population of component serial numbers, which many users opt not to do.

From the discussion above, I think we can enumerate two competing use cases:

1. I want unique inventory item names per device so that I can conveniently and predictable reference an item by its name.
2. A requirement for unique names would be burdensome for me because I often have no reason to name inventory items.

Is that a fair summary? And is there a case where you need to define a name for each item, but don't want it to be unique?

If not, a suitable compromise might be to make inventory item names optional (per #13776), but require names to be unique if defined.

[kkthxbye-code]

Is that a fair summary?

I would still like for component to be treated the same as parent in terms of uniqueness as they are essentially the same just for different target models. One refers to the related InventoryItem, another refers to the related device component.

I also think it should be taken into account how disruptive [*] of a change it will be (original PR or blank names). I don't know if you have a corpus you can check of netbox instances, but I imagine a rather large number of instances could have duplicate inventory item names which will all fail in the migration. Looking at the repo history, it's been possible to have duplicate named inventory items since 2016 v1.0.0 (called Modules back then).

Is the benefit of the change really worth it? Not sure what the benefit actually is, as netbox already supports making the names unique with custom validators. The only concrete reasoning that's been presented is that ansible doesn't work, which is usually not accepted as valid.

No one is forcing anyone to have duplicate named inventory items and there's ample oportunity to enforce this locally. On the other hand there's clearly a non-trivial number that find it useful and the only recourse is to skip the migration.

I think that's going to be my last input here, I'll leave it to the other to argument for their case.

[] The more strict uniqueness constraint on devices had a larger impact than expected and gave way for a large amount of issues/discussion/slack posts. I personally think this will be worse and it's slated for a patch release, not a minor release like the device change.*

[aaron-iles]

Is that a fair summary?

Yes I think that's fair. Up to you @jeremystretch .

[sartsj]

As a very recent addition to the netbox userbase, we have actually immediately started using the name as well for pretty much the SKU name, so like 'AMD EPYC 7282'. And for a two-socket system 2 of these items are assigned to a device.

Honestly I never even considered that names should be unique in this use-case, because to me that doesn't sound logical at all. The InventoryItem model already contains both asset tag and serial fields, which to me are perfectly suited to make part of the unique constraint should someone want/need that.

Otherwise, perhaps a 'Location Index' field (or whatever better name you can think of) would be more appropriate to add, which could then be used to identify (cpu) socket or fan slot? Perhaps this would have to be combined with jeremy's idea of making the field optional, but required to be unique if filled in? This would solve everyone's use-case, if serial and/or asset tag are not good enough.

[PieterL75]

I agree with @kkthxbye-code . Do not enforce uniqueness. If someone likes an inventory item name to be unique, then they can use custom validators.

[bitcollector1]

+1 do not enforce uniqueness for the name, the serial number is already unique if that is needed.

[jeremystretch]

As a very recent addition to the netbox userbase, we have actually immediately started using the name as well for pretty much the SKU name, so like 'AMD EPYC 7282'.

This is the purpose of the part ID field. Names are for the specific inventory item.

[sartsj]

As a very recent addition to the netbox userbase, we have actually immediately started using the name as well for pretty much the SKU name, so like 'AMD EPYC 7282'.

This is the purpose of the part ID field. Names are for the specific inventory item.

The documentation specifies 'The part identification or model number assigned by the manufacturer.' That would mean that in this specific example I would assume that to be '100-000000078', which I found on https://www.amd.com/en/products/cpu/amd-epyc-7282.

I mean, I'm not trying to be annoying here, but 'AMD EYPC 7282' is a product name, not a product id/part id/model number.

[jeremystretch]

It is a model number, and is intended to be conveyed by the part ID field. You're free to use the fields however you like, but this would not be justification for removing the uniqueness constraint on the name field.

[kkthxbye-code]

but this would not be justification for removing the uniqueness constraint on the name field.

I think I get what you mean, but it seems super disingeneous to suggest that anyone is advocating for the removal of the uniqueness check, as it makes it sound like we want a functional change. The uniqueness constraint has never been implemented properly since the very first public release of netbox. I get that this is a bug, but it doesn't change the reality the rest of the world lives in.

I'll close this out here as my initial suggestion has gotten no response and for some reason this has become a catch all feature request. I'll skip the migration and watch the fallout from afar. Anyone can feel free to create another FR for their preferred solution to this giant, huge, "rated low" severity issue.

Feel free to comandeer the issue and edit my top post, just make clear that it has been edited.

[jeremystretch]

Let's leave this open for a while to gather some more feedback. I'm confident we can identify a workable solution. It may not ultimately take the form of this FR, but I see no benefit in restarting the conversation at this stage.

[sartsj]

It is a model number, and is intended to be conveyed by the part ID field. You're free to use the fields however you like, but this would not be justification for removing the uniqueness constraint on the name field.

Apologies. Maybe my cpu example was not the best. I realize CPUs are an item category where the model number is part of the product name/description. However I do think it's important to state that there are item categories where this does not apply, and where it doesn't really make much sense to add additional numbering to the name field either. (SFPs, storage drives, RAM modules). I mean, which hard drive would be the 'first' drive, or which ram module would be the 'first' ?

I do agree there might be a 'nr 1' CPU in a system, but this is due to the socket it is in, not due to the CPU itself. This is why I suggested the location, or however you would want to call it. Although I suppose that might not be entirely appropriate either, since its not really a distinguishing feature of the cpu itself.

TLDR: What I am trying to say is that I do not think a name field is an appropriate way of adding uniqueness to a data model.

Anyway, I'll also hold off on further discussion as well, I think my point is clear anyway.

[josh-d-smith]

First, I’d like to say that it’s great that there’s such an active community that comes together to talk about issues/bugs/problems in such an open and constructive forum. Thanks everyone for the contributions here as I see many of you responding have created NetBox, created plugins, committed to this code base, or have contributed to this effort in some way shape or form.

Second, I think there’s absolutely no intent in the initial bug report of #13741 to break anyone’s implementation or interpretation of their NetBox deployment. The great thing about a useful tool is that it can be used in many ways. NetBox certainly wins the DCIM war because it’s so versatile and it’s why I like it.

Third, the topic of custom validators will definitely solve my use case for my environment with the goal of creating unique, deterministic, and programmable/API friendly Inventory Items.

Fourth, The reason I believed #13741 to be a bug was due to the documentation on Inventory Items stating the below. This implies that devices with a null parent should have a unique name which isn’t true at this moment. Maybe this bug report should result in improving/modifying the documentation instead of resulting in a code change??.

The inventory item's name. Must be unique to the parent device.

With the discussion so far, the community may benefit by having more examples and methods to tackle these problems and learn from each other. We all have our methods of solving problems and each has their own pros/cons. The goal is to find the method with the most pros and the least cons for each scenario.

The SFP scenario has been on my mind a lot recently because of the ability to split a single port into many ports when higher speed networking is involved.

• 100gb networking opticals can split into 2x 50gb interfaces or 4x 25gb interfaces (depending on the vendor)
• 200gb networking opticals can split into 2x 100gb interfaces, 4x 50gb interfaces, or 8x 25gb interfaces (depending on the vendor)
• 400gb (more splitting)
• 800gb (even more splitting)

On scenarios like the above, Inventory Items can only map to a single component (interface in this case) and the others are left hanging (I’m not sure if this FR solves this scenario). So what do we do then? Anyways, that’s food for thought.

As always, great discussion, and great community. Thanks everyone!

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. NetBox is governed by a small group of core maintainers which means not all opened issues may receive direct feedback. Do not attempt to circumvent this process by "bumping" the issue; doing so will result in its immediate closure and you may be barred from participating in any future discussions. Please see our contributing guide.

[jeremystretch]

As there hasn't been any further discussion in recent months, I'm going to close this out as stale. We might revisit it in the future.

[dsl81]

Was the constraint reintroduced in version 4 or it is a bug? I opened an issue but i did not get the answer.