opsmill / infrahub

Infrahub - A new approach to Infrastructure Management
https://opsmill.com/
GNU Affero General Public License v3.0
148 stars 7 forks source link

feature: ability to import device models from the netbox devicetypes repo #3672

Open ryanmerolle opened 3 weeks ago

ryanmerolle commented 3 weeks ago

Component

Not Sure

Describe the Feature Request

There is a wealth of device specifications we can pull from the netbox device type repo. We should be able to leverage this.

Describe the Use Case

Allows users to onboard device models more quickly.

Additional Information

No response

wvandeun commented 2 weeks ago

Here's an example of you can already achieve this today.

You will have to clone the https://github.com/netbox-community/devicetype-library/ repository locally, then set an ENV var NETBOX_DEVICETYPE_LIBRARY that points to the directory where the repo was cloned.

It's meant to be an example, it doesn't import everything at this stage and you need to use a more extensive schema.

We only tested this with some Juniper chassis based models, but it should be extendable to other models and manufacturers in the library.

import logging
import re
import os

from pathlib import Path

import yaml

from infrahub_sdk import InfrahubClient

DEVICETYPE_DIR = os.getenv("NETBOX_DEVICETYPE_LIBRARY", "") + "/device-types"
MODULE_BAY_MAPPING = {
    "FPC": {
        "regex": r"^FPC\s+(\d+)$",
        "kind": "InfraLineCardSlot"
    }
}

async def run(client: InfrahubClient, log: logging.Logger, branch: str, **kwargs):
    name = kwargs["name"]
    manufacturer = kwargs["manufacturer"]
    device_type = kwargs["type"]
    serial = kwargs["serial"]
    role = kwargs["role"]

    with Path(f"{DEVICETYPE_DIR}/{manufacturer}/{device_type}.yaml").open("r") as f:
        device_template = yaml.safe_load(f)

    device = await client.create(kind="InfraDevice", name=name, serialnumber=serial, type=f"{manufacturer}_{device_type}".lower(), role=role)
    await device.save(allow_upsert=True)
    await device.cardslots.fetch()

    slots_to_create = []

    for module_bay in device_template["module-bays"]:
        for key, mapping in MODULE_BAY_MAPPING.items():
            if re.match(mapping["regex"], module_bay["name"]):
                slots_to_create.append((mapping["kind"], module_bay["name"], module_bay["position"]))

    for kind, name, position in slots_to_create:
        position = int(position) #todo: ERROR handling
        slot = await client.create(kind=kind, name=name, position=position, device=device)
        await slot.save(allow_upsert=True)

You will need to use the following extended device schema

# yaml-language-server: $schema=https://schema.infrahub.app/develop/schema.schema.json
---
version: "1.0"
generics:
  - name: CableEndpoint
    namespace: Infra
    relationships:
      - name: connected_cable
        peer: InfraCable
        cardinality: one
        optional: true
        kind: Attribute
  - name: Interface
    namespace: Infra
    description: "Generic Network Interface."
    label: "Interface"
    display_labels:
      - name__value
    order_by:
      - name__value
    attributes:
      - name: name
        kind: Text
      - name: description
        kind: Text
        optional: true
      - name: speed
        kind: Number
      - name: mtu
        label: MTU
        default_value: 1500
        kind: Number
      - name: enabled
        kind: Boolean
        default_value: true
      - name: status
        kind: Dropdown
        optional: true
        choices:
          - name: active
            label: Active
            color: "#7fbf7f"
          - name: provisionning
            label: Provisioning
            color: "#ffff7f"
          - name: maintenance
            label: Maintenance
            color: "#ffd27f"
          - name: drained
            label: Drained
            color: "#bfbfbf"
      - name: role
        kind: Dropdown
        optional: true
        choices:
          - name: backbone
            label: Backbone
            color: "#6a5acd"
          - name: transit
            label: Transit
            color: "#9090de"
          - name: peering
            label: Peering
            color: "#ffa07a"
          - name: peer
            label: Peer
            color: "#faa446"
          - name: server
            label: Server
            color: "#98b2d1"
          - name: loopback
            label: Loopback
            color: "#93e9be"
          - name: management
            label: Management
            color: "#ccc28a"
          - name: uplink
            label: Uplink
            color: "#ff6b6b"
          - name: leaf
            label: Leaf
            color: "#e6e6fa"
          - name: spare
            label: Spare
            color: "#d3d3d3"
    relationships:
      - name: tranceiver
        peer: InfraTranceiver
        kind: Parent
        cardinality: one
        optional: true
        identifier: tranceiver__interface
      - name: linecard
        peer: InfraLineCard
        kind: Parent
        cardinality: one
        optional: true
        identifier: linecard__interface
      - name: device
        peer: InfraDevice
        kind: Parent
        cardinality: one
        optional: true
        identifier: device__interface
nodes:
  - name: Cable
    namespace: Infra
    include_in_menu: true
    label: Cable
    attributes:
      - name: identifier
        label: Cable identifier
        kind: Text
        optional: true
    relationships:
      - name: a_side
        peer: InfraCableEndpoint
        cardinality: one
        kind: Attribute
        optional: true
        identifier: cable__cable_endpoint_a
      - name: z_side
        peer: InfraCableEndpoint
        cardinality: one
        kind: Attribute
        optional: true
        identifier: cable__cable_endpoint_b
  - name: LineCardSlot
    namespace: Infra
    include_in_menu: true
    menu_placement: "InfraDevice"
    order_by:
      - position__value
    display_labels:
      - name__value
    attributes:
      - name: name
        label: Name
        kind: Text
        optional: false
      - name: position
        label: Position
        kind: Number
        optional: false
    relationships:
      - name: device
        label: Device
        cardinality: one
        optional: true
        peer: InfraDevice
        kind: Parent
        identifier: device__linecardslot
      - name: subslot
        label: Sub slot
        kind: Component
        peer: InfraLineCardSubSlot
        cardinality: many
        optional: true
        identifier: linecardslot__linecardsubslot
      - name: card
        label: Card
        kind: Component
        peer: InfraLineCard
        cardinality: one
        optional: true
        identifier: linecardslot__linecard
  - name: LineCardSubSlot
    namespace: Infra
    include_in_menu: true
    menu_placement: "InfraDevice"
    display_labels:
      - name__value
    attributes:
      - name: name
        label: Name
        kind: Text
        optional: false
    relationships:
      - name: parentslot
        label: Parent slot
        kind: Parent
        peer: InfraLineCardSlot
        cardinality: one
        optional: true
        identifier: linecardslot__linecardsubslot
      - name: card
        label: Card
        kind: Component
        peer: InfraLineCard
        cardinality: one
        optional: true
        identifier: linecardsubslot__linecard
  - name: Device
    namespace: Infra
    include_in_menu: true
    display_labels:
      - name__value
    attributes:
      - name: name
        label: Name
        kind: Text
        unique: true
        optional: false
      - name: serialnumber
        label: Serial number
        kind: Text
        optional: false
      - name: type
        label: Device Type
        kind: Dropdown
        optional: false
        choices:
          - name: juniper_mx960
            label: Juniper MX960
          - name: juniper_mx480
            label: Juniper MX480
      - name: role
        label: Device Role
        optional: false
        kind: Dropdown
        choices:
          - name: core_router
            label: Core Router
          - name: edge_router
            label: Edge Router
    relationships:
      - name: cardslots
        label: Card slots
        peer: InfraLineCardSlot
        cardinality: many
        kind: Component
        optional: true
        identifier: device__linecardslot
      - name: ports
        peer: InfraPort
        kind: Component
        cardinality: many
        identifier: device__port
        optional: true
      - name: interfaces
        peer: InfraInterface
        kind: Component
        cardinality: many
        identifier: device__interface
        optional: true
  - name: LineCard
    namespace: Infra
    menu_placement: "InfraDevice"
    include_in_menu: true
    display_labels:
      - type__value
      - serialnumber__value
    attributes:
      - name: serialnumber
        label: Serial
        kind: Text
        optional: false
        unique: true
      - name: type
        label: Type
        kind: Dropdown
        optional: false
        choices:
          - name: juniper_mpc7e-10g
            label: Juniper MPC7E-10G
          - name: juniper_mpc7e-mrate
            label: Juniper MPC7E-MRATE
          - name: juniper_mpc4e-3d-32xge-sfpp
            label: Juniper MPC4E-3D-32XGE-SFPP
    relationships:
      - name: ports
        peer: InfraPort
        cardinality: many
        kind: Component
        optional: true
        identifier: linecard__port
      - name: interfaces
        peer: InfraInterface
        cardinality: many
        kind: Component
        optional: true
        identifier: linecard__interface
      - name: slot
        label: Slot
        kind: Parent
        peer: InfraLineCardSlot
        cardinality: one
        optional: true
        identifier: linecardslot__linecard
      - name: subslot
        label: Sub slot
        peer: InfraLineCardSubSlot
        kind: Parent
        cardinality: one
        optional: true
        identifier: linecardsubslot__linecard
  - name: Port
    namespace: Infra
    label: Port
    menu_placement: "InfraDevice"
    include_in_menu: true
    display_labels:
      - name__value
    attributes:
      - name: name
        label: Name
        kind: Text
        optional: false
      - name: type
        kind: Dropdown
        optional: false
        choices:
          - name: 40gbase-x-qsfpp
          - name: 40gbase-x-qsfpp
          - name: 100gbase-x-qsfp28
          - name: 40gbase-x-qsfpp
          - name: 40gbase-x-qsfpp
          - name: 100gbase-x-qsfp28
          - name: 40gbase-x-qsfpp
          - name: 40gbase-x-qsfpp
          - name: 100gbase-x-qsfp28
          - name: 40gbase-x-qsfpp
          - name: 40gbase-x-qsfpp
          - name: 100gbase-x-qsfp28
          - name: 1000base-x-sfp
    relationships:
      - name: device
        label: Device
        peer: InfraDevice
        cardinality: one
        optional: true
        identifier: device__port
      - name: linecard
        label: Linecard
        cardinality: one
        kind: Parent
        optional: true
        peer: InfraLineCard
        identifier: linecard__port
      - name: tranceiver
        label: Tranceiver
        cardinality: one
        optional: true
        kind: Component
        peer: InfraTranceiver
        identifier: port__tranceiver
  - name: Tranceiver
    namespace: Infra
    menu_placement: "InfraDevice"
    include_in_menu: true
    display_labels:
      - serial_number__value
      - type__value
    attributes:
      - name: serial_number
        label: Serial number
        kind: Text
      - name: type
        label: Type
        optional: false
        kind: Dropdown
        choices:
          - name: qsfpplus
            label: QSFP+
          - name: qsfp
            label: QSFP
          - name: sfp
            label: SFP
          - name: gbic
            label: GBIC
    relationships:
      - name: port
        peer: InfraPort
        cardinality: one
        kind: Parent
        optional: true
        identifier: port__tranceiver
      - name: interfaces
        peer: InfraInterface
        cardinality: many
        kind: Component
        identifier: tranceiver__interface
  - name: InterfaceL3
    namespace: Infra
    description: "Network Layer 3 Interface"
    label: "Interface L3"
    icon: "mdi:ethernet"
    menu_placement: "InfraDevice"
    include_in_menu: true
    display_labels:
      - name__value
    order_by:
      - name__value
    inherit_from:
      - "InfraCableEndpoint"
      - "InfraInterface"
      - "CoreArtifactTarget"
    relationships:
      - name: ip_addresses
        peer: InfraIPAddress
        optional: true
        cardinality: many
        kind: Component
  - name: IPAddress
    namespace: Infra
    description: "IP Address"
    label: "IP Address"
    icon: "mdi:ip-outline"
    default_filter: address__value
    order_by:
      - "address__value"
    display_labels:
      - address__value
    include_in_menu: true
    attributes:
      - name: address
        kind: IPHost
      - name: description
        kind: Text
        optional: true
    relationships:
      - name: interface
        peer: InfraInterfaceL3
        optional: true
        cardinality: one
        kind: Parent