search

nautobot / nautobot-ansible

Ansible Collection for managing Nautobot Data
https://nautobot-ansible.readthedocs.io/en/latest/
GNU General Public License v3.0
44 stars 31 forks source link

REST API Inventory Better Handle Large Inventories #106

Closed jvanderaa closed 2 weeks ago

jvanderaa commented 2 years ago
ISSUE TYPE
SOFTWARE VERSIONS
pynautobot

N/A - REST API Inventory

Ansible:

x.x

Nautobot:

1.1.4

Collection:

networktocode.nautobot

SUMMARY

With large inventories, Ansible may cause requests to timeout for Nautobot due to asking for all resources.

STEPS TO REPRODUCE
  1. Create a large inventory in Nautobot
  2. Use Nautobot Inventory
  3. See timeouts and large CPU utilization
EXPECTED RESULTS

Completed inventory

ACTUAL RESULTS

Incomplete inventory

Possible Solution

  1. Look into handling pagination built into Nautobot instead of requesting all
  2. Evaluate moving to pynautobot to handle pagination
  3. Others
nniehoff commented 2 years ago

Another possible solution would be to investigate the use of GraphQL here.

jvanderaa commented 2 years ago

GraphQL inventory is the absolutely recommended methodology @nniehoff. We recommend keeping an inventory light and with only what is required to connect to the device. This way it is efficient and nimble. For example, not all playbook executions will need interface data. So why even have that in inventories. The playbook that needs to have the interface data should gather the data as needed from a SOT when required.

itdependsnetworks commented 2 years ago

There is pagination for device, but not the others, such as interfaces. While in general we should probably warn against using interfaces and many other data points, that being said, we should optimize in the same way in all scenarios.

jvanderaa commented 1 month ago

I'm going to get this covered with documentation. I'd rather not put a warning in that then people may try to supress.