[ For discussion ] Serializing runtime inventory back to native representation

[matman26]

I'm working on a project where I want to follow this kind of workflow:

1. Begin with a few devices in the network already known, specified as a SimpleInventory or registered on Netbox
2. Use CDP mapping to determine L2 neighbors within the Network

After running CDP mappings, I want to be able to:

3. Add more metadata to my existing and new hosts (i.e. under nr.inventory.hosts['somehost'].data = { ... })
4. Add the discovered hosts to my inventory (i.e. under nr.inventory.hosts['newhost'] = Host(...))
5. Do some tasks during runtime, possibly loop to map CDP neighbor for new entries and map the whole network
6. Once tasks are over, serialize all the new discovered hosts back to SimpleInventory YAML file or Netbox
   • Create new entries for hosts not already present in original source
   • Append whathever data was modified during runtime to new and existing hosts

In my view step 6 would enable some really interesting network as code workflows, but would require inventory plugins to expose some sort of API to INSERT runtime data back into their native representation (SimpleInventory would insert new discovered and updated hosts into origin YAML files, nornir_netbox would insert new hosts on Netbox as specified in its data model, etc.).

Of course, one can always dump the current in-memory data structure to a file or to netbox using pre-existing tasks, but enforcing (or at least providing) a sort of read-write mechanism for inventory plugins would facilitate this process from a user perspective and give inventory plugins this responsibility.

Any thoughts? Does this make sense?

[dbarrosop]

Hello, I am sorry but I am not sure I understand the issue here. The various points you raise are already supported. Did you mean to open this as an issue or as a discussion? I am closing this issue as I don’t think there is anything for us to do here but feel free to reopen otherwise.

[matman26]

It was meant as a discussion for a potential feature request. Could you elaborate on how step 6 is currently supported? From what I gathered from documentation, you can currently read inventory data from several sources via Inventory Plugins; what I'm proposing is that Inventory plugins could also have an "update" or "write" mechanism in order to get the current inventory representation in-memory and translate it back to the data source.

Let's suppose I began my workflow with the inventory (abridged):

# hosts.yaml file
R1:
  hostname: 192.168.0.1
  username: admin
  password: password

During runtime, I assume it's feasible to create hosts on-the-fly by manipulating nr.inventory, barebones example:

from nornir import InitNornir
from nornir.core.inventory import Host

# Initialize Nornir
nr = InitNornir(config_file='config.yaml')

# Add a new host
nr.inventory.hosts['R2'] = Host(name='R2', username='cisco', password='developer', hostname='192.168.0.2')

# Example of how 'serialization' would be used
nr.inventory.serialize()

After running something such serialize() (or whathever other name fits best), hosts.yaml should look like:

# hosts.yaml file
R1:
  hostname: 192.168.0.1
  username: admin
  password: password
R2:
  hostname: 192.168.0.2
  username: cisco
  password: developer

I guess one could call this a dynamic inventory implementation. The idea is to update the inventory during runtime and have changes reflect back on the source.

You CAN totally implement an equivalent of my proposed serialize() method by manually writing back to your inventory files (that's exactly what I'm doing right now and it works fine) during runtime, but I feel like having Nornir support an uniform interface that delegates this to Inventory plugins makes more sense from a design standpoint.

EDIT: If this seem like something useful, could the issue be reopened? I lack the permissions to reopen it myself without being a repo maintainer.