networktocode / ntc-netbox-plugin-onboarding

A plugin for NetBox to easily onboard new devices.
Other
245 stars 46 forks source link

interface detection+import logic question #127

Closed ckozler closed 3 years ago

ckozler commented 3 years ago

Hello - this is a wonderful open source project and the only reason I came back to netbox so thank you very much for your contribution

My question is about interface detection. On Juniper you have interfaces that get put on a vlan and set to access and then there is "integrated routing bridge" (irb) that handles getting the IP address. Taking it a step further, individual interfaces can be put in a LAG and then the VLAN's are attached to the LAG

xe-0/0/32. xe-1/0/32 -> ae0 LAG -> irb.2

As 2 stands for the VLAN tag ID number

The reason I explained all of that is for context to my question - when I ran the onboarding process the process only detected irb.2 meanwhile there are technically 3 other active interfaces (xe-0/0/32, xe-1/0/32, ae0) that still need to exist on the device so that functions like wire tracing can work. It makes it a little confusing if everything goes up to an irb as it doesnt show A/Z side termination points

In this example I could show xe-0/0/32 on switch A goes to xe-0/0/10 on switch B

I flipped through the code to try to see if I can see what is glossing over these other active interfaces and I believe its https://github.com/networktocode/ntc-netbox-plugin-onboarding/blob/0a92fdb346974ea5a7b5e1e1704cdd816efcf879/netbox_onboarding/netdev_keeper.py#L260

get_interfaces call to NAPALM does return interface status like is_up so that could be used to decide what is imported?

The interfaces are also detected in get_facts but I dont know if up/down status is there. So my thought is import everything that is found in get_interfaces and then call get_interfaces_ip then compare and then update those devices just onboarded

[14:41:04]root@myhots:~]# napalm --vendor junos -u root -p redacted --optional_args keepalive=600,auto_probe=600,timeout=600 10.0.0.3 call get_interfaces | jq 'to_entries[] | "(.key), (.value | .is_up)"' | grep "(xe-[0-9]/0/3[0-3]|xe-1/0/31.16386|irb.2)" "xe-0/0/32, true" "xe-1/0/31, false" "xe-1/0/32, true" "xe-0/0/32.0, true" "xe-1/0/31.16386, false" "xe-1/0/32.0, true" "irb.2, true"

Whereas calling _ip -

[14:42:48]root@host:~]# napalm --vendor junos -u root -p redacted --optional_args keepalive=600,auto_probe=600,timeout=600 10.0.0.3 call get_interfaces_ip | jq 'to_entries[]' | grep "(xe-[0-9]/0/3[0-3]|xe-1/0/31.16386|irb.2)" "key": "irb.2",

mzbroch commented 3 years ago

Check out Network Importer tool by @dgarros - https://github.com/networktocode/network-importer - I hope this one should fit for your use case.

As for the described behaviour - onboarding will only add a management interface and it is an intended behaviour.

Additionally, you can modify plugin's default behaviour and implement your own logic with onboarding extensions - your code will be executed by an onboarding plugin, while you will also have an access to opened Napalm device object - check documentation for details.

ckozler commented 3 years ago

OK that was my misunderstanding then. I thought it would use NAPALM to find all interfaces with and without IP's and assign those IP's to the interface in netbox but are you saying it essentially finds the first interface or IP and makes that management IP in netbox? If so, in this example, irb.2 is not the management interface but onboarding assigned it as such. Or is expected behavior to find only interfaces with IP's. If the latter, why not find all devices? I am just curious, not pressing for anything

Thank you for those additional details

dgarros commented 3 years ago

@ckozler currently the onboarding plugin will create only the management interface and the management interface will be identified based on the ip address provided to connect to the device in the first place. In your case if irb.2 has been identified as the management interface, I would assume it's because the ip address that you provided or the ip address associated with the dns name that you provided is configured on this interface. If that's not the case, that we should investigate that.

mzbroch commented 3 years ago

Onboarding plugin uses the given IP to login into the device and fetch device's details. If the option create_management_interface_if_missing is used, plugin will also add device's interface (only one) under this IP address is configured along with IP address object and marking this IP as primary IP of the onboarded device. This interface, will not have its status in NetBox changed to "management only"

As I mentioned, you can control 100% of what is created in NetBox through onboarding extensions. You can re-use NetBox Keeper logic and use it to manage the onboarding your way.

To answer your question regarding import of all interfaces - another tool has been designed and created to control data sync and import - network-importer. A lot of other objects gets involved into process - VLANs, Trunks, LAGs, IP Addresses, etc.

ckozler commented 3 years ago

Thanks @mzbroch

@dgarros on juniper, the real management interface is me0, vme0, and fxp0 (I am forgetting one or two). Those are, by definition, the management interfaces. I say by definition because with Juniper, the management interface is on an unrouted IP address. So if 10.0.0.3 is my management IP I cannot reach it from my netbox host at 10.100.100.25 Its not that its routing specific, it just sees the incoming IP as on a different network and drops the packets

So in this case, since NAPALM only supports 3 major device OS', wouldn't the onboard logic be better more helpful to assign management based on the interface names? Just my $0.02 from my first run with this today

irb.2 is not my management interface. Its the switch IP for that VLAN so I can reach it from other networks and is more of an "access IP" than a true management interface

dgarros commented 3 years ago

@ckozler, ok I think the issue is that we have a different definition of management interface.

For the onboarding plugin, we don't really care what is the management interface on the device we are just looking for the primary IP that should be used to communicate to a device. That we call the management interface is whatever interface is associated with the IP we used to communicate to the device.

NetBox has a concept of management interface but we are not using it and the interface that we are creating is not created as a management interface. Hope that helps

ckozler commented 3 years ago

Thanks for clarifying @dgarros

If I'd do a merge request, would you consider if I implemented a feature such as that I describe?

My idea is that it would be complimenting the already existing onboarding feature - i.e separately from what you already do today, it would be a new enhancement to also find/detect the management interface found by NAPALM