Add support for tracking physical cable plants

[jeremystretch]

Currently, NetBox does not support any concept of physical cable plant installations. We can connect one interface to another, but there is no way too associate that connection with information about the physical path taken by the cabling.

Ideally, NetBox should support the construction of physical cable paths to be associated with data (and perhaps console) connections. This would entail the tracking of individual path panels and cables, as well as the ability to support bundles of cabling (e.g. MPO fiber trunks).

The database schema necessary to support this feature will take quite a bit of thought. However, once that schema has been devised and agreed upon, it should be relatively easy to incorporate in NetBox.

[shakalandy]

Nice proposal. Please keep in mind that patch panels could be ordered horizontally or vertically. This would be a great enhancement.

[doon]

This is where I've always run into roadblocks in designing the schema, especially when looking at some of the cross connect panels we have here. Rack 1, 2U Fiber Shelf in pos41,42 6 Cassettes (Duplex LC) and each cassette goes to a different Rack / Room, etc. So the schema kept getting hairy. couple that with input / output / the ability to connect to either a interface, another patch panel, or say a circuit / provider it snowballs pretty quickly.

[puck]

With @doon's comment about things snowballing quickly. My thought would be to treat anything that a cable can terminate on as a port. Each socket on a patch panel has two ports, one on the back for structured cabling, and one on the front for patching (and yes, I'd treat a punch down block as a port). A cable can have one or more plugs on each end. Those plugs can go into ports. If you need to connect two cables together it might be able easier to have a coupler that has two (or more) ports on it.

This kind of model should also support octopus cables, with mismatching number of plugs (think MTP on one end and many LC on the other).

Cables could have metadata with them about the maximum speed they support, colour etc.

This model also extends to power infrastructure where PDUs can be treated as patch panels. DC bus bars etc.

[mdomore]

For a patch panel i create a device type and i insert 1 interface front and one interface rear for each FO patch in the patch panel.

i connect rear from patch 1 to rear from patch 2 and front to active devices.

the only big problem is the form factor.

it's a trick but if it can help others it's free

[isodude]

@mdomore Nice trick, I figured I would solve it that way as well, however, I loose the connection between server -> switch in that case since it is just a connection in terminology.

I would like a socket-port, possibly a hub, since that's exactly what it is :) That would mean instead of add a 10G SFP+ interface I add a Hub instead where you can connect several interfaces. Maybe that will also shorten the timespan for designing the models since we don't need to invent that much around it.

[isodude]

I also noted that it's not possible to add new Interfaces, if we could focus on that part, maybe this is just a matter of the SysOp to define them and specify how many connections a interface can have.

Today there's not a XenPak connections i.e., also not 10GE Copper. That would be nice be able to add yourself.

[mdomore]

Yes and if we can add connector type like LC, SC, SCA, LCA ... this could be really usefull

[brianaddicks]

Something else that I think goes along with this feature is tracking the usage of a patch panel. It'd be great to see that on the Rack list next to the RU Utilization %. That way if I have a rack with lots of free physical space but no patch panel room I won't accidentally tell someone it's available to use.

[swickstrom]

My current bodge for this is to create a parent device which acts as a copper/fiber panel, and then create child devices that serve as copper/fiber cassettes. Each cassette has ports and there is a crosslink interface under the OOB interfaces that connects the cassette pairs. The cassettes are mirrors of eachother as far as port assignments go; if you wish to determine where a device on port-01 traces to you follow the crosslink port to the cassettes mate and see which device is on port-01. The issue with this idea revolves solely around the binding action of interface types. I specifically ran into issues representing console/serial connections; due to how console interfaces work, I had to have the console ports on the cassette in the rack with the console server be OOB ports, and the console interaces in the rack with the servers act as console server ports in order for the ports to be assignable. Enter the passthrough form-factor. A passthrough form-factor would be defined as: • Having an in and an out interface assignement • Having no interface type; allowing it to bind with any interface and not require pre-requisites such as being a console server port or being an OOB port

[amatamalas]

We are currently using openDCIM and one of the point to start using in the past was the ability to track cabling (but missing the IPAM feature like netbox has). Basically you have devices with data ports (network, console,...) and power ports (to connect to your PDU). The Patch panels are the only special object which have front and back port in order to be able to represent all the patching path. Then, each connection can have a label and you can represent the full path of a connection by searching by one of the labels of any cable in the path. I think this is really useful when you need to dive into the datacenter. The form factor, cable type (ethernet, fiber, coax,...) or even the color of the cable are very good attributes (optional and configurable) to have into account. I'm not a fan of reinventing the wheel and maybe it could be a good starting point for the cable plan design, as it's working very well for us. Of course there are quite a lot of usage cases and maybe this is too simple to cover all of them.

[iamdadmin]

One way you could accommodate single connections would be to allow edit the device types and devices forms to support "back-to-back connection" in the same way that you have "Power Ports" and "Interfaces" only this type of connection has an A side and a B side.

The A side can be connected to a fibre bundle such as Device: "48 CORE BUNDLE 4728" which in turn can have back to back connections. In this way you'd eventually describe the entire path.

[doon]

I've been toying with this on my own (in ruby/rails as I not so good with Parseltongue :)).

What I've come up with, and it seems to work so far...

ports belong to a PortGroup, which then belongs to a PortGroupOwner, these are basically abstractions to represent anything that can have a "port". So for example.

PortGroupOwner == ASR9K PortGroup (index: 0) GigabitEthernet0/0/0 Ports(0) GigabitEthernet0/0/0/0 Ports(1) GigabitEthernet0/0/0/1

or

PortGroupOwner = RandomDellServer PortGroup (index: 0) Mezzanine Ethernet Card Ports(0) Eth0 Ports(1) Eth1

or even an abstract entity such as a hand off to another carrier,

PortGroupOwner = RandomCarrier PortGroup(index:1) 80SS Handoffs Port(0) TieCircuit XYZ

Then each circuit consists of port connects (cross connects).

this connection has a port_a and a port_b (Which are both FKs into ports) and both are required. the circuit_id is also required unless this connection is marked as permanent.

port_a and port_b have unique constraints on them. So each port can only be used once as port_a and once as a port_b.

Permanent port_connects are used between panels, and are generally created at panel creation time. I have scripts that build perm connections between 2 port_groups with the same number of ports to automate panel creation).

So when assigning to a circuit I just need to create connections between 2 ports.(service tries to create it as a,b, and then b,a, and both violate constraints it fails). if either ports belongs to a perm cross connect, then it updates the circuit_id for the existing port_connection for those ports, else it just creates the new one and assigns the circuit id.

Once we have all the ports mapped out in the circuit you can find the endpoints by taking the set of ports as A ports and the set of ports as B ports and look for which elements are exclusive.

To graph it can just dump it into dot/graphviz as the the series of cross connects.

to build a DLR or the like can just start at one of the ends and follow the port connects til you reach the other end.

So far it seems to solve most of the issues/requirements that I have.

Hope this helps, or feel free to tell me I Am crazy/misguided/etc..

[harsh-kotak]

[Suggestion] You can start with adding colums line this in the interface connections menu: A-END -- Z-END Row/Rack DeviceA Interface Media -- Row/Rack DeviceB Interface Media

where media is cable/connector type. e.g.: SM LC, MM LC, QSFP twinax, copper, etc.

This does not map the entire Layer 1 but helps to keep track of cabling type used.

[nward]

Something that would be useful is to treat fibre pairs as individual fibres, or bundle them together. Some providers track every strand, rather than pairs. This would also enable use of BiDi optics, etc.

MPO/MTP support would be very useful, also. I currently model MTP connected patch panels in Device42 by pretending that there are many panel-to-panel cables, which isn't right obviously!

Perhaps support for inline circulators and taps, too. A company I work with splices these inline, rather than going through a rack mounted device.

[RyanBreaker]

I'm glad to see this is on the roadmap to be implemented as this is the most-needed feature for us to switch from a spreadsheet for tracking all our cable paths. Is there an ETA or milestone for when this will be implemented?

[Jluisito]

+1000! Great feature to a great software. It will have a diagram of the connections like Opendcim does?

Thanks

[quentindavid]

Hi, this feature is the only reason I cannot use this software at the moment !

May I help you to achieve this ?

[th3goose]

I've been spending a lot of time prior to finding netbox thinking about cable and pathway design and documentation. There are really three main parts to this, paneling with ports, cabling and "pathways" which would include not just the cabling but the duct and construction types being aerial, buried, riser, etc, as well as geo information that would be used to create a geo based map from the lat/lon attributes of each part of the pathway. Cable is more of a logical construct in this methodology (including a relationship with the pathway its a part of), but the most important part is the pathway concept that basically lays the groundwork for what your trying to achieve, similar to a circuit.

@jeremystretch can you provide anymore color to what your thinking here with panels, cables, etc? I'd be interested in helping depending on the need for this.

[jeremystretch]

I don't think we'll extend beyond representing physical cables and patch panels in NetBox. This by itself is already fairly complex, especially when you consider that cable types can change between the front and back of a panel. Attempting to map the actual physical path is more of facilities management thing, and well outside the scope of NetBox.

[th3goose]

@jeremystretch I think thats fine, how would you suggest a custom module be created to interface with netbox all the while limiting the forking of code and continually having to merge branches? Since netbox is its own branch I'm having a hard time wanting to fork it to create a custom module and losing future fixes from the main branch.

[afics]

I need support for fanout cables, for example QSFP+ to 4xSFP+.

[VictorJ76]

Any news about patch panels ?

[hyberdk]

+1 here, fiber patch panels are a must for me (so I can connect 2 racks together).. else it looks fantastic..

bounty?

[oasys]

Registering my support/interest.

This is a critical feature for us.

[RyanBreaker]

The same for our site, we're looking for a system for tracking specifically this as well as the other features in Netbox.

[VictorJ76]

Same for me. Critical feature, this is one of the main uses of netbox for us.

[nward]

2 ways to make sure stuff gets done in open source - do it yourself, or sponsor someone (the project/a third party) to do it.

I don't have time for the former right now. There's enough people on here that if we all threw in some $ it'd be a reasonable donation. I'd put in a few hundred I reckon. It'd save me paying for device42 or whatever...

[isodude]

I love the netbox concept. Not having proper cable plants is the reason we can't use it.

I believe we're also interested in putting in cash for solving this issue. How do we proceed?

Any developer that are interested in looking at this feature? I understand that this could easily be a mess if done incorrectly?

[RyanBreaker]

I'd be willing to put the development time in myself, just not sure where to start with this. I'll look around the code later in my free time to see the options on where to best proceed.

[jsenecal]

I did some work on this kind of use case on a project I was working on in a previous company. It happened to be a Django project so if Jeremy feels like it maybe we could start working on the feature and submit PRs?

Did you already have something in mind?

On Tue, Apr 4, 2017, 12:05 Ryan Breaker, notifications@github.com wrote:

I'd be willing to put the development time in myself, just not sure where to start with this. I'll look around the code later in my free time to see the options on where to best proceed.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/digitalocean/netbox/issues/20#issuecomment-291548353, or mute the thread https://github.com/notifications/unsubscribe-auth/ABfg5o0YE8MhR6uiVyE-Wkkvr7T0dnglks5rsmo3gaJpZM4I-GO2 .

[mkbt]

The company I work for will sponsor development time on this also if needed.

I take it the schema design hasn't quite made it yet and the first step would be organising schema/design then having it signed off by @jeremystretch ?

[RyanBreaker]

@jsenecal Sorry for late reply, I'm not as active on Github as I used to be. I didn't have any immediate ideas and was hoping to contribute to any development attempts for this mostly because I'm just unsure where to start. But as soon as anyone is able to share a roadmap or more solid ideas on how to get started I'd be much more able to jump in.

@mkbt That would be FANTASTIC. Please do share updates and anything anyone else can do to contribute.

[beggles]

Would a starting point be to show basic relationships between equipment racks, patch frames (cross connects), media type, and network / storage switches? For instance;

Cabinet X has copper cabling to patch frame A, and Fibre cabling to cross connect B, and needs to interface a host via copper to Network Segment Y.

Network Segment Y is available on a variety of physical switches, though not all switches are available via patch frame A. In order to "join the dots", we would therefore need to record the particular patch frame where each switch is loomed to.

This method assumes that the equipment rack chosen has cable plant available, and that the outcome sought is to "simply" identify a switch candidate to attach the host to - and that there would be a validation of port availability, against the candidate switch's current run state.

[beggles]

So I've done some more thinking on this and have the following to add;

Physical elements to "start" with would be;

• Patch Panels / Switch Loom (cable distribution point - device attachment)
• Cross Connect Frame / Head of Row (cable aggregation point)

A patch panel may be associated to one rack or many (consider consolidation points). A switch loom could be associated to an individual switch or module within a larger switch.

A cable (or datapoint) interconnects a Patch Panel / Switch Loom to a Cross Connect Frame. Similarly, a cable (or datapoint) attaches a device, be that server or network, to another device.

Cable would be further categorised as Fibre (MM/SM) Copper (Cat 5e/6/6a). Similarly Patch Panels would be categorised as Fibre or Copper, you may want to add the presentation interface type for Fibre, ie LC/SC/MPO etc, though this may have limited mileage.

That should mostly take care of the physical layer, and I should add that Patch Panels being cabled to Cross Connect Frames would not be exclusive, Patch Panels can connect to other Patch Panels, just as Cross Connects Frames to other Cross Connect Frames.

As this would go toward some sort of database schema, you should then be able to search for cable capacity between Patch Panels and Switch Looms (Switches) for instance, and in turn recommend the "path" to patch through to get the desired connection.

Finally, further to my original post, a switch (loom) could then be associated to a Network Segment definition (or multiple, such as Corporate/Extranet/Management etc), thus allowing Netbox to "suggest" how to provision the desired "service" based on Equipment Rack where a host is to be installed, and its proximity to a particular Network Segment, as mapped through the above "structure".

[puck]

Sounds good.

With the cable interconnects, remember that one cable may go to one or more ports on the patch panel. For a cable which goes to more than one port on a patch panel, it may only go to one port on the other end.

[beggles]

@puck, so I think you are referring to something like an MPO trunk into a Fibre cassette? If so, I was more thinking of a "cable" being a discrete mapping to a physical data point on a one to one basis, rather than mapping pairs into a multi-core trunk, which for my requirements would be more than sufficient. To expand upon this, if I was to install a 12 port LC cassette into a patch panel I would add 12 cables to that, rather than 2 MPO trunks - in more concerned with physical capacity / capability, if that makes sense.

[nward]

It is reasonably common for a LC->MPO cassette to be installed, where the MPO cable goes to an MPO-only patch panel. Additionally, it is common for MPO connected equipment (like 100G and 40G multimode) to be split across many single fibres in to a DWDM transponder. It is common for a 100G or 40G interface with MPO connectors to be split in to pairs of 10G and treated like individual 10G interfaces (look at modern 1RU switches with only QSFP28 holes, for example).

Additionally, MPO cables have multiple different polarities - straight through (A), rolled cable (B), and swapped pairs (C). These have impacts on all of the above items.

I think MPO needs to be modelled, with a lot of flexability.

Single fibre working is common also, so I think fibre needs to be modelled with pairs (and multicore like MPO) being templates for grouping.

I think also that copper should be modelled as individual wires too - what about those environments where someone sticks one of those 2:1 Ethernet cable sharers in place to save a few pennies? Or a cable is rolled for connection to a serial console? RJ45 1:1 should be a template of 8 cores. It should be possible to model other constructs and trace every single electrical connection through between devices.

[nward]

Patch panels are also interesting - many patch panels are modular. Some vendors make the mistake of assuming patch panels have only one level of modules (or worse, no levels!) I have seen patch panels which accept multiple levels.

I would also like to see fibre storage drawers modelled, so it can be documented where a fibre's slack is taken up. This might be a bit tricky though so is a nice to have!

[beggles]

@nward at the simplest level if you represent a "data point" as a single point of connect, and then in turn assign it a media type, ie. copper outlet, fibre outlet, MPO trunk etc. Based on the media type you "could" then define a number of connection types (both pre and user defined), where you could then "allocate" the individual conductors/cores, which in turn could be "fanned" out to multiple devices, consider where using all four pairs on a copper data point to deliver multiple modem lines, or an MPO fan out fable to attach multiple 10Gb interfaces across a number of hosts - this would also then require, as an option, the ability to define each core/conductor of a data point.

I think the first step in "solving" the problem is to "map" out a structure for the physical layer, and then define connection types per each media type - if that makes sense?

[nward]

I don't really understand what you're saying, sorry.

Let me take a stab at it, it sounds like we might be roughly on the same page here.

I think the best way to model this is to model things at the lowest level, and group them together (i.e. multiple copper cores in to a single cat6 cable, or two fibre cores in to a fibre pair) with some information about polarity (i.e. cores 1/2 in a fibre pair are swapped).

Perhaps most users only look at things at that higher level, and perhaps the initial implementation doesn't even expose the lower level information, but, modelling that up front is in my view the best approach.

[beggles]

For the most part, yes, I would expect that users would in the first instance treat a "data point" in a patch panel as nothing more than a place to patch something into, like a server for instance - so by default a given cable is nothing more than a point of connection, irrespective of its core/conductor count, basically a means of getting from point A to point B, and sometimes via point C.

Where there is a need to further "divide" a cable into its cores/conductors, this would be done by means of defining a "connection type", where the default is 1-1, or maybe "host". The connection type characteristic would then be mapped "over" the cable, and in turn inform the active characteristic of the cable, for both A and B ends, if other than the default "host" connection characteristic.

[nward]

From what I can tell, you're attempting to model this from an arbitrary level of the stack - i.e. what you consider to be a "normal" cable, and apply characteristics to it if it is not normal (though perhaps "normal" is a characteristic in itself). That is going to drive a lot of complexity for situations we can't foresee right now (though I am fairly certain that with any data model based on this approach, I can foresee problems with by looking at environments I work in which have some fairly complex cabling at times).

Let's model it from fundamental components individual cores/conductors, and group things in to digestible constructs like "cables" for those who don't need that level of detail. I imagine most users will only care about "cables" - be it a 8-core cat6 with straight-through RJ45 termination, or, a 2-core fibre pair, or whatever - however, having that level of detail underneath means: 1) telcos, and other environments with slightly unusual cabling can be modelled today. 2) users who have a "cable" view of the world who subsequently find that they need to split out a single pair can do so simply by reaching down the stack without re-modelling things.

[beggles]

I don't know about you, though in my environment each cable is presented in a "standard" configuration / polarity. An RJ45 outlet for instance is terminated to the 568A standard, and any "alteration" to that is made by what you attach to it - i.e. where we need a serial console, our terminal server is configured to present the "roll-over", or where we need an ethernet crossover, we use a crossover lead.

The cable at its most basic level is a nothing more than a mechanism for connecting endpoints together, irrespective of whether they are serial consoles, ethernet, voice etc, for copper and in a similar manner for fibre.

Maybe I'm over simplifying it, though coming from a cabling background, there is pretty much nothing you can't do with a cable... :)

[nward]

In "telco" world we document cores. Copper is typically documented to the "pair" level, but even that can occasionally end up with weird things happening.

It's more and more common for enterprises to buy dark fibre - here in NZ it's very cheap to do so. This is provided as single fibres, not pairs. This is often presented on a patch panel as an LC pair, however.

So, now in a "cable-centric" model, within your cabinet you can run a single cable which is a pair - which is easy. However, you need to have a "cable" which is actually two physical cables out the back, which are two seperate physical circuits which may take different paths through a city. They will almost certainly be spliced at different places.

So, how do you model that? Do you create a special single-use cable that somehow has different circuit identifiers for each fibre? What if they each go to different devices at the far end - perhaps CWDM muxes? I'm sure it's do-able, but, are we just figuring out chains of workarounds?

I think the best approach is to be able to provide the average user with an interface that lets them connect "cables" just like you want - but under the hood model that in much more detail, so that a user can drill down and document individual cores/conductors how they see fit to match their environment - remembering of course that not everyone does things the "right" way, and there are a lot of legacy/hacks around the place.

[nward]

Taking things one step further, having indefinite "layers" of encapsulation would be helpful here also.

This would allow us to document, for example the following layers for a cable between two buildings:

• duct
• tube
• multi-core fibre
• fibre strand
• colour

In this way, we can trace connectivity between two devices even through complex topologies.

Perhaps this allows modelling of active circuits - i.e. perhaps a telco provides us a single port over which we have multiple services (perhaps differentiated by VLANs). Those services may terminate in different remote locations - or perhaps different sets of remote locations, in the case of a multi-point service.

I guess it depends how far up the stack we want to go - but providing indefinite layers here would no doubt help.

[beggles]

I like the idea of "encapsulation", which you could use to as you say layer "characteristics" to a cable. In the "telco" world how would you name / identify internal vs external cable routes?

[shepherdjay]

We're also looking for this feature to manage patch connections across our mdf to idfs. Coming from Issue #912 where @jeremystretch specifically discourages using devices to represent patch panels.

[deggler]

We too would be very interested in that feature. It would be helpful to know how many fibres are available between to patch panels.

[ghost]

Any plugins or something available for this solution? That's the feature I need for our management and I thought it's build into the DCIM as I installed it. Two days later I run straight into this feature missing.

Do you know when it will be implemented?

[flokli]

@alsternerd For me, the problem seems to be that it's currently not clear how the cable tracking part should look like, on the data models and feature side.

There were various people here willing throw coding time on implementing it, but as previously written by @mkbt, first a concecpt/schema needs to be done and signed off by @jeremystretch