Multi-termination no longer working in web UI since 4.0

TheWidowLicker commented 4 months ago

Deployment Type

Self-hosted

NetBox Version

v4.0.2

Python Version

3.11

Steps to Reproduce

Click on a front port or interface Create a cable and connect to other interface/frontport Try to add second interface (you should be able to have multiple terminations)

Expected Behavior

That you should be able to have multiple termiantion points

Observed Behavior

Each time you type in the second device interface it just replaces the orignhal temination point. You can only have 1. Old "cables" made from 3.7.x still show the multi terminations but it you edit them and add or edit a new termination point, all others are wiped out and you have jsut 1 termination point

TheWidowLicker commented 4 months ago

I upgraded to 4.0.2 to see if this would fix the issue but it appears to still be the same. I cannot choose more than I parent termination point. I can choose multiple interfaces or front ports if they are on the same device. Once you try to add a 2nd device/interface the 1st is removed/overwritten

arthanson commented 4 months ago

@TheWidowLicker thank you for opening a bug report. I was unable to reproduce the reported behavior on NetBox v4.0.2. Please re-confirm the reported behavior on the current stable release and adjust your post above as necessary. Remember to provide detailed steps that someone else can follow using a clean installation of NetBox to reproduce the issue. Remember to include the steps taken to create any initial objects or other data.

Can you please expand on the steps to reproduce - I added interfaces on two devices and connected a cable between them and added a second interface to the cable (see screenshot) it added and saved. I'm not sure if I'm following the same steps you went through however.

Monosnap Editing cable #137 | NetBox 2024-05-20 07-30-45

TheWidowLicker commented 4 months ago

Hi Thanks for this but the issue isn't what you have above. The issue is that I cannot pick 2 interfaces, 1 on each switch/host. So imagine, I have a front port on a patch panel and the fiber cable from this (tx side) is split/tapped and goes to multiple servers. The servers have say PHY14 and the split TX cable goes to the RX side on the PHY14's on the servers. I was able to do this multi-termination (multiple servers to 1 front port). I have also tried connecting multiple NIC's/interfaces (1 from each server) to a switch port (which I was able to do on 3.7.x), and I get the same situation. Once I try to add the second host/system and its interface, it over writes the 1st instead of adding it. I have some old cables, created before the upgrade and they still have the multiple host interfaces to 1 interface/front port. I dont want to edit them in case I lose these. Thanks

For Clarity: here is a cable made before upgrade. You can see 3 host interfaces (rx side) connects to 1 front port panel (TX side) Cables-before-upgrade

Here is a cable I made after. I can only seem to connect to 1 host interface. If I try to add a 2nd host/NIC I lose the first Cables-after-upgrade

TheWidowLicker commented 4 months ago

If its just something im doing wrong, please let me know. Thanks

TheWidowLicker commented 4 months ago

I also tried to make this happen in the public demo and again. I can have 2 ports from the same device but not 1 port from 2 devices https://demo.netbox.dev/dcim/cables/132/

TheWidowLicker commented 4 months ago

I noticed that there is a fix in 4.0.1 for an issue that was discovered with cable termination. Maybe this is related to that, even though that specific issue appear to be fixed in 4.0.2 Issue https://github.com/netbox-community/netbox/issues/15973

arthanson commented 4 months ago

@TheWidowLicker can you please provide a step-by-step repro, something like:

add a device A
add interface i1 to device A
add device B
add front-port P1 to device B ...

It can take a lot of time to figure out and reproduce issues if the reproduction steps aren't clear.

TheWidowLicker commented 4 months ago

Her goes. In this video I create a cable from an interface on a host, then try to add a second host with interface (this removes the 1st host and interface). I then try to add the original back (this removes the second host and puts back the 1st). Finally I show a cable made before the upgrade that has 3 hosts (and there interfaces) connected to 1 front port. 1 host only allowed.webm

I found a strange work around. If I add interfaces without choosing a host I can add multiple hosts (unfortunately as it doesn't let me see the hosts at time of picking its hit or miss on getting it right) Work around but shows no device names.webm

I don't seem to be able to upload pictures or videos. I will try from another location

cademetz commented 4 months ago

@arthanson

I am having the same issue. For context, in my specific case, I am trying to create a cable connection to represent a QSFP breakout cable: directly from a switch QSFP port to four different servers' SFP ports (no panels in between).

Consider these steps:

Create three devices: sw-1, svr-1, and svr-2
Create three interfaces, one per device created in step 1, calling all interfaces int-1
Create a cable connection. For Side A, select interface for the type, select sw-1 for device, select int-1 for the interface. For Side B, select interface for the type, select svr-1 for the device, and int-1 for the interface.
Attempt to add svr-2, int-1 to Side B.

This is where the issue will present itself.

Attempt mechanism 1:

Attempt to type int-1 in the Interface field for Side B (in essence, searching for svr-2, int-1). No results will be found. Presumably, this is because the results are filtered to only svr-1, (as selected in Step 3 above). This therefore seems to prevent the user from selecting an interface from any other device once a device is populated in the Device field of Side B (or Side A for that matter).

Attempt mechanism 2:

Attempt to type svr-2 in the Device field of Side B. It will be shown, but if it is selected, svr-1(and its interface) will be removed from the connection.

Attempt mechanism 3 (this is the work around @TheWidowLicker mentioned):

(If svr-1 is still listed in the Device field for Side B) Remove svr-1 from the Device field of Side B (click the 'x' icon in the field); this should remove the interface as well.
Without selecting a device, search for the int-1 in the Interface field. All int-1 results will be listed. The user CAN select the int-1 interface for svr-1 and svr-2 here. The problem is that it is unknown which int-1 you are selecting. In this specific example, we only have two int-1's from which to select (belonging to svr-1 and svr-2) so we accomplish our goal. However, in a larger environment where interface names may be duplicated many times over, it will not be possible to ensure you select the right "int-1" for the intended device. In my specific environment, with numerous ESXi hosts, I have many interfaces called "vmnic0" and no way of telling them apart, unless the interfaces have had unique descriptions provided to them somehow indicating to which device the interface belongs.

TheWidowLicker commented 4 months ago

@arthanson have you been able to recreate the issue @cademetz and I are having.

TheWidowLicker commented 3 months ago

Is it possible to get the severity increased on this. It was a working featurw pre 4.x and now doesnt work. Its also a very useful part of netbox amd I would really like to be able to use this to showcase to colleagues the features of netbox

molusk commented 3 months ago

Hello, I am having the issue too. We are using this feature to be able to match physical cables with functional network using blade center and PCIe cards shared with the blades. The cable is from the patch panel and the blade center PCIe card but the card is "plugged" to a blade so we create the card twice : once in the blade center module bay, once in the blade module bay and double the termination to be able to trace the cable to the actual server connected. If we change the association between the card and the server, we keep the physical cable and destroy the second termination, this is very handy !

But since version 4, we can't select interfaces from different devices. As soon as we select the second device, the interface field is emptied... I wonder if we can import multi-termination cable via CSV as a workaround ? @TheWidowLicker's workaround is unmanageable for me.

molusk commented 3 months ago

I was able to make @TheWidowLicker's workaround work for us but it breaks the whole interest of multi-termination cables in our case : we have first to delete the termination before being able to select it again, it is not possible just to add a termination. As soon as you empty the device field the interface disappears but if you look for it, it is greyed out and not selectable until you save and destoys the termination in the db. So, I had to create a dummy device to move my termination without losing the topology and recreate the multi-termination cable at once.

jeffgdotorg commented 3 months ago

Bumped severity to medium since the workaround is pretty unwieldy and potentially breaks other workflows.

TheWidowLicker commented 3 months ago

Any update on this. Any idea whats causex it or even a usable workaround

jeffgdotorg commented 3 months ago

My understanding is it's strictly a frontend issue, and that data model and API support for multi-termination remain working. A usable workaround may be possible by operating the API. Playing the odds, I would speculate that one of the many UI updates introduced during 4.0 development is the proximate cause of the breakage.

We will get this bug fixed in a 4.0.x patch release, but I can't predict exactly when. Thanks for your patience, and for your help in getting the issue reproducible in the lab.

TheWidowLicker commented 3 months ago

Thanks for the reply. Im not used to working with API. I will ha e to tey amd see 8f I can find an example of a multi-termination between 2 hosts amd see if I can edit it to suit my case.

sol1-matt commented 2 months ago

I have a customer that wants multi cable termination so I'm looking into this. I'm a little unsure I'm looking at the correct cause, @TheWidowLicker and @cademetz are you able to confirm I've identified the issue correctly here.

My steps to reproduce in Netbox 3.6.7

Create 3 devices, test[1-3]
Create interfaces on each device, testif
In test1 testif add a cable connecting to test2 testif

I can't add a new cable on test1, only edit the exiting cable I can add a new cable on test3 testif but I can't connect that to test1 testif as the testif is grey'd out

But I can edit the cable on test1 testif and remove device b then find (guess) the interface for test3 testif and I now have multiple connections.

In the API the cable object has a_terminations and b_terminations which are lists and in my example the list for the B side has 2 objects in it.

Is this the correct behaviour? Multiple connections on a single cable object.

The screen shot in https://github.com/netbox-community/netbox/issues/16176#issuecomment-2121997548 and #15948 Seems to suggest that it is the correct behavior pre Netbox 4.

I can get this working in Netbox 4.0.5 as well.

The difference is in 3.6.7 I can use the search feature to find test2 then add it's interface, then use it again to search test3 and add it's interface. When I search test3 the interface for test2 isn't removed.

In 4.0.5 the same procedure removes the interface of test2 when you search and select test3.

Is the problem in 4.0.5 the removal of interfaces already added to a cable that aren't related to the current device when searching and then selecting for other devices to add interfaces from?

I'll note the only way I got this to work in 4.0.5 was because I created interface names that were globally unique to the test case then skipped searching for devices, I just did a interface search and found 3. It wouldn't have worked if I had more than a handful of interfaces with the same name.

TheWidowLicker commented 2 months ago

@sol1-matt You are correct in that on netbox 4.x when you search and add the 2nd TestIF it removes the first. Therefore you cannot do mutiple cables to interface/s. It is supposed to be possible by using the API (i.e. just a frontend bug) but I am not familiar with using API calls and have not found any example API documentation that would show how to do this. I did install and tinker with Postman but havent go it yet. If you know of any docs that show how to do multi-termination or have an example I would appreciate it, as I really want to setup the cabling and fix is still not available yet.

sol1-matt commented 2 months ago

@TheWidowLicker It is "technically" possible without the api by leaving the device blank and searching for the interfaces names only. But unless you have globally unique interface names it is practically impossible.

I can confirm it is frontend issue related to the device fields on the page, the bug is triggered when you clear the device field or return results to the device field which first clears it. It looks like listeners are setup on page load for "device clear" to clear not only the device field but also the interface field.

I believe the solution to be simply making the device clear only clear the device and not the interface but I'm not quite sure where in the code this is getting set. @jeffgdotorg, @arthanson a Netbox dev familiar with this area of Netbox's codebase would likely be able to point to where this is controlled from off the top of their head.

To get the data into Netbox without the GUI you should be able to use the the API, look at the API doc's for cables, as that is what we will be creating. https://netbox/api/schema/swagger-ui/ and search for "dcim/cables/", it is the post.

While you can't use bulk import to create the multi terminated cables the I find the bulk import page a excellent reference for the values I want to use, particularly the blue question marks.

The other really useful feature in Netbox 4 are ID's in the object views, to get the data for the api go to https://netbox/dcim/interfaces, add ID through Configure Table and find the interfaces you want to link together in there.

your api request will look something like this

curl -X POST "http://netbox/api/dcim/cables/" \
-H "Authorization: Token <your-api-token>" \
-H "Content-Type: application/json" \
-d '{
    "type": "",
    "a_terminations": [
        {
            "object_type": "dcim.interface",
            "object_id": 24998
        }
    ],
    "b_terminations": [
        {
            "object_type": "dcim.interface",
            "object_id": 24999
        },
        {
            "object_type": "dcim.interface",
            "object_id": 25000
        }
    ],
    "status": "connected",
    "label": "long cable",
    "color": "FF00FF",
    "length": 40075,
    "length_unit": "km",
    "description": "really long cable",
    "comments": "seriously long cable"
}'

If you want to use different object type, like front port, you just need to add the right values, the object type will always match the objects url components, eg: https://netbox/dcim/interfaces = dcim.interfaces, https://netbox/dcim/front-ports/ = dcim.front-ports.

davekempe commented 2 months ago

so @sol1-matt if you rename the interfaces for multi-terminations, then perhaps you can do it in the UI?

sol1-matt commented 2 months ago

so @sol1-matt if you rename the interfaces for multi-terminations, then perhaps you can do it in the UI?

Yes, if the interfaces have globally unique names they can be configured in the UI because you can find them without setting a device.

Resetting or clearing the device appears to be the cause of this problem, so if they aren't reset or cleared it allows you to create multi termination cables fine.

TheWidowLicker commented 1 month ago

I attempted to do this via ansible using this playbook `---

name: Connect PHY14 Rx interfaces from three servers to a single front port in NetBox hosts: localhost gather_facts: false tasks:
- name: Ensure NetBox API token is available ansible.builtin.assert: that:
  - netbox_api_token is defined fail_msg: "NetBox API token is required"
- name: Get device details for server1 netbox.netbox.netbox_device: netbox_url: "{{ netbox_url }}" netbox_token: "{{ netbox_api_token }}" data: name: FRAEGDTHOPS573
  
  site: FRAE
  
  role:
  
  device_type: "Dell PowerEdge R650"
  
  register: server1
- name: Get device details for server2 netbox.netbox.netbox_device: netbox_url: "{{ netbox_url }}" netbox_token: "{{ netbox_api_token }}" data: name: FRAEGDTHOPS574
  
  site: FRAE
  
  role:
  
  device_type: "Dell PowerEdge R650"
  
  register: server2
- name: Get device details for server3 netbox.netbox.netbox_device: netbox_url: "{{ netbox_url }}" netbox_token: "{{ netbox_api_token }}" data: name: FRAEGDTHOPS575
  
  site: FRAE
  
  role:
  
  device_type: "Dell PowerEdge R650"
  
  register: server3
- name: Print out server details ansible.builtin.debug: msg:
  - "{{ server1 }}"
  - "{{ server2 }}"
  - "{{ server3 }}"
- name: Get interface details for server1 PHY14 Rx netbox.netbox.netbox_device_interface: netbox_url: "{{ netbox_url }}" netbox_token: "{{ netbox_api_token }}" data: device: "{{ server1.device.name }}" name: "PHY14 Rx" register: PHY14_Rx_server1
- name: Get interface details for server2 PHY14 Rx netbox.netbox.netbox_device_interface: netbox_url: "{{ netbox_url }}" netbox_token: "{{ netbox_api_token }}" data: device: "{{ server2.device.name }}" name: "PHY14 Rx" register: PHY14_Rx_server2
- name: Get interface details for server3 PHY14 Rx netbox.netbox.netbox_device_interface: netbox_url: "{{ netbox_url }}" netbox_token: "{{ netbox_api_token }}" data: device: "{{ server3.device.name }}" name: "PHY14 Rx" register: PHY14_Rx_server3
- name: Print out interface details ansible.builtin.debug: msg:
  - "{{ PHY14_Rx_server1 }}"
  - "{{ PHY14_Rx_server2 }}"
  - "{{ PHY14_Rx_server3 }}"
- name: Get front port details netbox.netbox.netbox_front_port: netbox_url: "{{ netbox_url }}" netbox_token: "{{ netbox_api_token }}" data: device: "PP:0205:1208629" name: "FP 3B2 TX" type: lc rear_port: "RP 3B2 TX" register: FP_port
- name: Print out connection details ansible.builtin.debug: msg:
  - "{{ FP_port }}"
- name: Connect PHY14 Rx of server1 to front port netbox.netbox.netbox_cable: netbox_url: "{{ netbox_url }}" netbox_token: "{{ netbox_api_token }}" data: termination_a_type: dcim.interface termination_a: device: "{{ server1.device.name }}" name: "{{ PHY14_Rx_server1.interface.name }}" termination_b_type: dcim.frontport termination_b: device: "PP:0205:1208629" name: "{{ FP_port.front_port.name }}" register: connect_server1
- name: Connect PHY14 Rx of server2 to front port netbox.netbox.netbox_cable: netbox_url: "{{ netbox_url }}" netbox_token: "{{ netbox_api_token }}" data: termination_a_type: dcim.interface termination_a: device: "{{ server2.device.name }}" name: "{{ PHY14_Rx_server2.interface.name }}" termination_b_type: dcim.frontport termination_b: device: "PP:0205:1208629" name: "{{ FP_port.front_port.name }}" register: connect_server2
- name: Connect PHY14 Rx of server3 to front port netbox.netbox.netbox_cable: netbox_url: "{{ netbox_url }}" netbox_token: "{{ netbox_api_token }}" data: termination_a_type: dcim.interface termination_a: device: "{{ server3.device.name }}" name: "{{ PHY14_Rx_server3.interface.name }}" termination_b_type: dcim.frontport termination_b: device: "PP:0205:1208629" name: "{{ FP_port.front_port.name }}" register: connect_server3
- name: Print out connection details ansible.builtin.debug: msg:
  - "{{ connect_server1 }}"
  - "{{ connect_server2 }}"
  - "{{ connect_server3 }}"`

The the first connection is fine but when it tried to add the interface from the second host to the same front port I get this error. TASK [Connect PHY14 Rx of server2 to front port] **** fatal: [localhost]: FAILED! => {"changed": false, "msg": "{\"all\":[\"Duplicate termination found for dcim.frontport 405: cable 87\"]}"}

Not sure if this is related to the above issue but it looks like I can do multi-termination using the netbox ansible module

TheWidowLicker commented 1 month ago

Also if ansible modules just run API calls then doesn't this mean I cant use API to create multi-termination as suggested above. That would make this seem like a backend issue rather that webui issue.

I might be missing something though

TheWidowLicker commented 1 month ago

I added in a task to get the details of a working cable (made before moving to 4.x) Here is the output of the not working termination/cable/connection and then the working one. Only real difference I can see is that the working one shows 3 link_peers (numbers are the id of the server interfaces) Unfortunately it doesn't look the ansible module lets you call link peers as a parameter { "changed": false, "failed": false, "front_port": { "_occupied": true, "cable": 90, "cable_end": "B", "color": "", "created": "2024-05-17T07:47:01.207986Z", "custom_fields": {}, "description": "", "device": 962, "display": "FP 3B2 TX", "id": 405, "label": "", "last_updated": "2024-07-25T14:21:10.603856Z", "link_peers": [ 26160 ], "link_peers_type": "dcim.interface", "mark_connected": false, "module": null, "name": "FP 3B2 TX", "rear_port": 1611, "rear_port_position": 1, "tags": [], "type": "lc", "url": "http://devtpsnbox801.ire.susq.com:8000/api/dcim/front-ports/405/" }, "msg": "front_port FP 3B2 TX already exists" }, { "changed": false, "failed": false, "front_port": { "_occupied": true, "cable": 15, "cable_end": "B", "color": "", "created": "2024-05-17T07:47:01.210071Z", "custom_fields": {}, "description": "", "device": 962, "display": "FP 3D5 TX", "id": 431, "label": "", "last_updated": "2024-05-17T07:47:01.210077Z", "link_peers": [ 1411, 1429, 1447 ], "link_peers_type": "dcim.interface", "mark_connected": false, "module": null, "name": "FP 3D5 TX", "rear_port": 1637, "rear_port_position": 1, "tags": [], "type": "lc", "url": "http://devtpsnbox801.ire.susq.com:8000/api/dcim/front-ports/431/" }, "msg": "front_port FP 3D5 TX already exists" }

sol1-matt commented 1 month ago

I'd be inclined to use a curl command or python/pynetbox over ansible, the ansible netbox functions don't always expose the full functionality of the Netbox API and tend to lag slightly behind releases as well.

brandan-schmitz commented 1 month ago

Anyone know if this is on a roadmap anywhere to get fixed? I updated to v4.0.8 this morning and the issue is still present. We have many of these types of cables and trying to teach people how to create the API requests through something like CURL is not easy to do when they do not have experience with that type of thing.

Creating cables is a core function of Netbox and the ability to create these types of cables easily should be an important thing to get fixed!

We have also found that trying to edit an existing cable will cause the same issue so that is also causing issues for us!

TheWidowLicker commented 1 month ago

@TheWidowLicker It is "technically" possible without the api by leaving the device blank and searching for the interfaces names only. But unless you have globally unique interface names it is practically impossible.

I can confirm it is frontend issue related to the device fields on the page, the bug is triggered when you clear the device field or return results to the device field which first clears it. It looks like listeners are setup on page load for "device clear" to clear not only the device field but also the interface field.

I believe the solution to be simply making the device clear only clear the device and not the interface but I'm not quite sure where in the code this is getting set. @jeffgdotorg, @arthanson a Netbox dev familiar with this area of Netbox's codebase would likely be able to point to where this is controlled from off the top of their head.

To get the data into Netbox without the GUI you should be able to use the the API, look at the API doc's for cables, as that is what we will be creating. https://netbox/api/schema/swagger-ui/ and search for "dcim/cables/", it is the post.

While you can't use bulk import to create the multi terminated cables the I find the bulk import page a excellent reference for the values I want to use, particularly the blue question marks.

The other really useful feature in Netbox 4 are ID's in the object views, to get the data for the api go to https://netbox/dcim/interfaces, add ID through Configure Table and find the interfaces you want to link together in there.

your api request will look something like this
curl -X POST "http://netbox/api/dcim/cables/" \
-H "Authorization: Token <your-api-token>" \
-H "Content-Type: application/json" \
-d '{
    "type": "",
    "a_terminations": [
        {
            "object_type": "dcim.interface",
            "object_id": 24998
        }
    ],
    "b_terminations": [
        {
            "object_type": "dcim.interface",
            "object_id": 24999
        },
        {
            "object_type": "dcim.interface",
            "object_id": 25000
        }
    ],
    "status": "connected",
    "label": "long cable",
    "color": "FF00FF",
    "length": 40075,
    "length_unit": "km",
    "description": "really long cable",
    "comments": "seriously long cable"
}'
If you want to use different object type, like front port, you just need to add the right values, the object type will always match the objects url components, eg: https://netbox/dcim/interfaces = dcim.interfaces, https://netbox/dcim/front-ports/ = dcim.front-ports.

I was able to create these using your curl command which I put into ansible, so thanks for that. I do agree with @brandan-schmitz that this should be quite an important fix as I believe it is fundamental to using netbox. I will continue with the curl commands for now

jeremystretch commented 1 month ago

As a reminder, this has been tagged needs owner. It's just waiting for a volunteer.

sol1-matt commented 1 month ago

@jeremystretch I had a dig through the code previously and commented above

I believe the solution to be simply making the device clear only clear the device and not the interface but I'm not quite sure where in the code this is getting set. ... a Netbox dev familiar with this area of Netbox's codebase would likely be able to point to where this is controlled from off the top of their head.

If somebody in your team is able to point me in the right direction for where this would be controlled I'd be happy to have another crack at it, I wasn't able to grok the codebase enough to figure it out on my own.

arthanson commented 1 month ago

Code is in https://github.com/netbox-community/netbox/blob/develop/netbox/dcim/forms/connections.py#L13 Issue is it needs to query with the device Id (like here: https://github.com/netbox-community/netbox/blob/develop/netbox/dcim/forms/connections.py#L38) so it only handles one device at a time. There is a UI aspect as before there was no distinction which device the interfaces was from in a multi-termination case. Am thinking potentially having the device be multi-select and then a sub-field for each one selected. Something like:

Type:                   [Interface]
Device:               [dev1, dev2]
Interface dev1:  [   ]
Interface dev2:  [   ]

TheWidowLicker commented 1 month ago

I'm no good at coding so can't advise on above but am just wondering if we could diff the code for that section you provided and see what the change was (as it was working in 3.x series). If this is a stupid suggestio, apologies but again I don't know any programming

arthanson commented 1 month ago

Decided to go with a multi-select on the first field, and add the device/circuit/etc.. to the name in the sub-selector:

Editing cable #1 | NetBox 2024-08-07 14-16-09

netbox-community / netbox