Add option to get primary_ip DNS name

[v0tti]

We usually use the DNS name (FQDN) of the device as the name of the host in Icinga. This does not seem to be possible at the moment, because the DNS name is associated with IP addresses in NetBox. I could not figure out how to import DNS name information with this module. It would be great if it could be added (or explained if it is already possible).

[davekempe]

Hi, this is already supported, but I can see how its a little confusing.

Here is an importsource as a director basket:

{
    "ImportSource": {
        "Netbox - Device - Sol1 FW": {
            "description": null,
            "key_column": "nbsol1fw_name",
            "modifiers": [
                {
                    "description": null,
                    "priority": "1",
                    "property_name": "primary_ip4",
                    "provider_class": "Icinga\\Module\\Director\\PropertyModifier\\PropertyModifierArrayElementByPosition",
                    "settings": {
                        "position": "address",
                        "position_type": "keyname",
                        "when_missing": "null"
                    },
                    "target_property": "ip4_cidr"
                },
                {
                    "description": null,
                    "priority": "2",
                    "property_name": "ip4_cidr",
                    "provider_class": "Icinga\\Module\\Director\\PropertyModifier\\PropertyModifierRegexReplace",
                    "settings": {
                        "pattern": "\/\\\/.*\/",
                        "replacement": ""
                    },
                    "target_property": "ip4_address"
                },
                {
                    "description": null,
                    "priority": "3",
                    "property_name": "site",
                    "provider_class": "Icinga\\Module\\Director\\PropertyModifier\\PropertyModifierArrayElementByPosition",
                    "settings": {
                        "position": "slug",
                        "position_type": "keyname",
                        "when_missing": "null"
                    },
                    "target_property": "site_slug"
                },
                {
                    "description": null,
                    "priority": "4",
                    "property_name": "tenant",
                    "provider_class": "Icinga\\Module\\Director\\PropertyModifier\\PropertyModifierArrayElementByPosition",
                    "settings": {
                        "position": "slug",
                        "position_type": "keyname",
                        "when_missing": "null"
                    },
                    "target_property": "tenant_slug"
                }
            ],
            "originalId": "1",
            "provider_class": "Icinga\\Module\\Netbox\\ProvidedHook\\Director\\ImportSource",
            "settings": {
                "apitoken": "xxxxxxxxxxxxxxxxxxxxxxxxxx",
                "baseurl": "http:\/\/netbox.example.com\/api",
                "filter": "tag=sol1-firewalls&status=active",
                "flatten": "",
                "mode": "1",
                "munge": "s=nbsol1fw,name",
                "proxy": ""
            },
            "source_name": "Netbox - Device - Sol1 FW"
        }
    }
}

The sync rule just takes the ipv4_address and maps it to the address. For non-primary IPs it might be a little more plumbing.

[v0tti]

I do not see how this solves my issue. This gives me the primary_ip as an address for Icinga, but I want the DNS name that is associated with that IP and use it as the name for the host.

[davekempe]

sorry I misunderstood the question. That will require a little more digging, I will get @sol1-matt to take a quick look.

[sol1-matt]

This is a curly one @v0tti

The device or vm object contains basic IP info, pretty much the same as any other linked object in Netbox, which is why we don't know much about it.

 "primary_ip": {
        "id": 1234,
        "url": "https://netbox/api/ipam/ip-addresses/1234/",
        "display": "10.9.2.2/24",
        "family": 5,
        "address": "10.9.2.2/24"
    },

Where as a full ip object contains everything about the IP except for the fact it is a primary IP.

Normally I'd be looking at doing something like importing the IP's separately and making them host templates then linking the host and template together. The problem with this approach is without being able to filter on 'is a primary IP' there could be a lot of IP's that aren't needed which becomes messy and could end up with a large query (expensive) and lots of data we don't need.

The alternative is looking up the primary ip from the device/vm at import time, each primary ip is another api call. So if you are importing 1000 devices you go from 20 API calls to 1020.

One alternative I've used is a custom field FQDN on the device/vm object. Not a big deal in the situation I used it in as the use case didn't record IP's, as they were dynamic, just the FQDN which had the associated dynamic IP updated in dns when it changed. So I wasn't duplicating netbox functionality because and the name was associated correctly, device/vm <- dns <- ip rather than device/vm <- ip <- dns.

When I think about this an ideal situation would be for Netbox to have dns as it's own object that could be linked to devices, virtual machines and ip addresses but that isn't going to help us now.

I don't know if a custom field is a suitable workaround for your situation.

I'll look into getting the dns from the primary IP and importing IP's, but they are likely to be very expensive options and will need to both be optional.

[sol1-matt]

I've added ip address as a import type, I haven't tested this yet but it does mean you could pull the ip addresses and create host templates with a dns value if you really want.

Using this is likely to want to use the a salted id, munge = s=nbip,id = nbip_{ip id} , as the object name. I think there would be drama handling dots and slashes in object names.

---

Looking at adding the primary ip to the device/vm after reviewing the code and adding the above.

Getting dns is not dissimilar to getting services for devices, we want a linked netbox object merged. Services on devices are implemented by getting the filtered list of devices/vm's, getting the list of all services them adding any services to the matching device.

Looking at a netbox installation which I think is typical we have 3,000 devices, 11,000 ip addresses and 600 services. I did look at ip addresses that only had a interface (so none that weren't attached to a device) but that barely changed the total for the data set I was looking at.

The reason I think this is typical is that services are only added on some devices/vm when they are required, the limiting factors here are a) only required and b) added to a device or vm. IP's on the other hand can be added even if they aren't linked anything. There are also likely to be more ip's in a network than devices or services.

If we accept these figure the api load generated by get all devices and services is ((3000 (devices) / 1000 (devices per pagination)) x 10 seconds (example time)) + ((600 (services) / 600(devices per pagination)) x 8 seconds (a less full pagination has overhead so is less efficient)) = 4 api calls and 38 seconds which is about right.

If we do the same for device + all ip's ((3000 (devices) / 1000 (devices per pagination)) x 10 seconds (example time)) + ((11000 (ip's) / 1000(devices per pagination)) x 10 seconds (example time)) = 14 api calls and 140 seconds as an estimate, substantially longer to process.

If we try this by looking at only devices that have a primary ip and requesting the ip data from the API on a per ip address basis we get this ((3000 (devices) / 1000 (devices per pagination)) x 10 seconds (example time)) + (2000 (devices with primary ip making a individual api call for the ip address) x 0.1 seconds (example time)) = 2003 api calls and 230 seconds.

---

Either way is expensive, but I think getting all ip addresses is less expensive because anybody who wants to use dns names from ip addresses is likely to use primary ip and dns in ip more than somebody that puts the fqdn in the device/vm name or on a custom field.

And if we are going to go to the effort of parsing all ip's with a assigned interface because we can't filter on ip's that have a primary ip only then we may as well add all the non primary ip linked to a given device too

primary_ip_address:{ 
  primary ip details here
},
all_ip: [
  {
     ip 1 details here
  },
  {
     ip 2 details here
  }
]

though parsing arrays in icinga is a pain if what you want is something like just the dns names. There is also the inevitable exploding of data.

I'm going to think on adding all ip's for devices for the time being, ip is a lot more than address and dns, there is role, tags, vrf and status all of which could be useful even a linked vlan if you dig.

[v0tti]

Thanks for the detailed answer and the quick implementation! I am still a bit stuck because I do not know how to assign a particular IP to an host.

[v0tti]

A colleague of mine has figured it out. We now have an import source for IP-Addresses. A sync rule puts the IP ID and the DNS name in a data list. In the import source for the devices we have modifiers to get the ID of the primary IP and than map it to the data list to get the DNS name. If there is nothing you want to further discuss, I think we can close this issue.

[sol1-matt]

Interesting solution, similar to what I was suggesting with the host template but without the mess of a bunch of templates that won't be used. It is neater, I like it.