Unable to filter based netbox custom field in dict format

[walidamer711]

Along with the host I receive a custom field using netbox inventory in dict format.

'vlandomain': {'value': 1, 'label': 'dc1'},

I am not able to filter based on this attribute, I have tried F filter with nested option as the following but it seems that work only when this information is attached to dict called "data" to search inside.

inv = nr.inventory.filter(F(vlandomain__value=1))

I got this error: AttributeError: 'NoneType' object has no attribute 'get'

however, I can access this directly from the host. In [28]: inv.hosts['rtr1'].get('vlandomain').get('value')
Out[28]: 1

[dbarrosop]

it seems that work only when this information is attached to dict called "data" to search inside

in 2.0.0 that's the only place where user data can exist, but that's handled by the plugin itself so you don't have to worry about it.

Which version of nornir are you running?

Can you also run the following code?

for host, data in inv.hosts.items():
    print(f"{host}")
    print(data.get("vlandomain").get("value"))

I suspect the issue is that some host may not have that "vlandomain" data but I need to confirm before writing a patch.

[walidamer711]

for host, data in inv.hosts.items():
    print(f"{host}")
    print(data.get("vlandomain").get("value"))

I suspect the issue is that some host may not have that "vlandomain" data but I need to confirm before writing a patch.

I am running 2.0.0 version. You are right not all the hosts have this "vlandomain" set and I got error for hosts that don't have this data when I run above code. AttributeError: 'Inventory' object has no attribute 'inventory'

[dbarrosop]

After some testing I can only reproduce your problem if some host has vlandomain = None. Is that your case?

[dbarrosop]

If I understood finally your issue (my previous comment) following PR should fix your issue: #272

Could you install from that branch and test? Thanks!

[walidamer711]

After some testing I can only reproduce your problem if some host has vlandomain = None. Is that your case?

Yes this is the case. Other hosts have None value for "vlandomain". 'vlandomain': None,

[walidamer711]

If I understood finally your issue (my previous comment) following PR should fix your issue: #272

Could you install from that branch and test? Thanks!

I'll test and confirm. Thanks

[dbarrosop]

I merged the PR so I will be closing this issue, please, if it doesn't solve your issue reopen it and let us know. Thanks!

[walidamer711]

I used to initialize netbox inventory with the following in ver. 2.0.0: nr = InitNornir(num_workers=100, inventory="nornir.plugins.inventory.netbox.NBInventory", NBInventory={"nb_url": "http://xxxx", "nb_token": "xxx"}) but in this branch "fix-270", I initialize as the following: nr = InitNornir(core={"num_workers": 100}, inventory={"plugin": "nornir.plugins.inventory.netbox.NBInventory", "options": {"nb_url": "http://xxxx", "nb_token": "xxxx"}}) and I got these errors for all netbox custom fields for all hosts: hosts -> N5K_DC_ACC_01 -> dc_vlan_domain extra fields not permitted (type=value_error.extra) hosts -> N5K_DC_ACC_01 -> has_fex extra fields not permitted (type=value_error.extra) hosts -> N5K_DC_ACC_01 -> vlandomain extra fields not permitted (type=value_error.extra)

[walidamer711]

Is it documented how to initialize netbox inventory in version 2.0.0-beta? I am not able to find it.

[dbarrosop]

What you are doing looks correct so you may have found a bug (that's why it's in beta :) ). Let me look at it. In order to reproduce and fix, would you mind sending me a sample of the call?

GET $NB_URL/api/dcim/devices/?limit=0

I need to see the json blob returned by that query (feel free to modify private/important information).

[walidamer711]

it will be a huge json file if I attach 41 devices in the call. So, I have attached here two sample devices, one with custom filed set and one with None. { "id": 1, "name": "N5K_DC_ACC_01", "display_name": "N5K_DC_ACC_01", "device_type": { "id": 1, "url": "http://url/api/dcim/device-types/1/", "manufacturer": { "id": 2, "url": "http://url/api/dcim/manufacturers/2/", "name": "Cisco", "slug": "cisco" }, "model": "N5K-C5548UP", "slug": "n5k-c5548up" }, "device_role": { "id": 15, "url": "http://url/api/dcim/device-roles/15/", "name": "DC Access", "slug": "dc-access" }, "tenant": { "id": 2, "url": "http://url9/api/tenancy/tenants/2/", "name": "MZA-Infra", "slug": "mza-infra" }, "platform": { "id": 2, "url": "http://url/api/dcim/platforms/2/", "name": "Cisco NX-OS", "slug": "nxos" }, "serial": "", "asset_tag": null, "site": { "id": 2, "url": "http://url/api/dcim/sites/2/", "name": "MV1", "slug": "mv1" }, "rack": { "id": 1, "url": "http://url/api/dcim/racks/1/", "name": "DCC12", "display_name": "DCC12" }, "position": 20, "face": { "label": "Rear", "value": 1 }, "parent_device": null, "status": { "label": "Active", "value": 1 }, "primary_ip": { "id": 2, "url": "http://url/api/ipam/ip-addresses/2/", "family": 4, "address": "" }, "primary_ip4": { "id": 2, "url": "http://url/api/ipam/ip-addresses/2/", "family": 4, "address": "" }, "primary_ip6": null, "cluster": null, "virtual_chassis": null, "vc_position": null, "vc_priority": null, "comments": "", "tags": [], "custom_fields": { "vlandomain": { "value": 1, "label": "mv1_dc" }, "dc_vlan_domain": true, "has_fex": true }, "created": "2018-10-01", "last_updated": "2018-11-05T07:01:52.213888Z" },{ "id": 10, "name": "3750_OOB_EXT_SW", "display_name": "3750_OOB_EXT_SW", "device_type": { "id": 15, "url": "http://url/api/dcim/device-types/15/", "manufacturer": { "id": 2, "url": "http://url/api/dcim/manufacturers/2/", "name": "Cisco", "slug": "cisco" }, "model": "WS-C3750-24TS", "slug": "ws-c3750-24ts" }, "device_role": { "id": 10, "url": "http://url/api/dcim/device-roles/10/", "name": "OOB Switch", "slug": "oob-switch" }, "tenant": { "id": 2, "url": "http://url/api/tenancy/tenants/2/", "name": "MZA-Infra", "slug": "mza-infra" }, "platform": { "id": 1, "url": "http://url/api/dcim/platforms/1/", "name": "Cisco IOS", "slug": "cisco_ios" }, "serial": "", "asset_tag": null, "site": { "id": 2, "url": "http://url/api/dcim/sites/2/", "name": "MV1", "slug": "mv1" }, "rack": { "id": 2, "url": "http://url/api/dcim/racks/2/", "name": "DCC1", "display_name": "DCC1" }, "position": 24, "face": { "label": "Front", "value": 0 }, "parent_device": null, "status": { "label": "Active", "value": 1 }, "primary_ip": null, "primary_ip4": null, "primary_ip6": null, "cluster": null, "virtual_chassis": null, "vc_position": null, "vc_priority": null, "comments": "", "tags": [], "custom_fields": { "vlandomain": null, "dc_vlan_domain": null, "has_fex": false }, "created": "2018-10-08", "last_updated": "2018-11-04T06:40:22.275963Z" } please let me know if you need more information.

[dbarrosop]

The json blob you pasted is not valid, would you mind double checking the parts you removed to make sure you didn't leave sections half-cut?

[dbarrosop]

You can use this to validate your json and inspect it looks as you expect:

https://jsonlint.com/?json=

Thanks!

[dbarrosop]

Ok, no need for that. Taking a quick look at the current code it was easy to spot the issue and was able to reproduce.

Can you test this PR?

https://github.com/nornir-automation/nornir/pull/274

The commit fixing your issue is: https://github.com/nornir-automation/nornir/pull/274/commits/8b9641578d691a3208b9d1a38ff68e2c83d7cfd7

[walidamer711]

Wow. I have tested with branch "inv_plugins_data" and retrieving the hosts from netbox with custom filed is working perfectly fine as well as the filtering with F filter with nested option.

Thanks

[dbarrosop]

Second beta with support for this has been released.