ytti / oxidized

Oxidized is a network device configuration backup tool. It's a RANCID replacement!
Apache License 2.0
2.78k stars 920 forks source link

Duplicated nodes using http pagination #3138

Open stevesaner opened 5 months ago

stevesaner commented 5 months ago

I am using Netbox as a source for oxidized. I am on the latest version of oxidized and here is the source config (oxidized is running on the same machine as netbox)

http: url: http://127.0.0.1:8001/api/dcim/devices/?status=active&cf_oxidized=true scheme: http delimiter: !ruby/regexp /:/ headers: Authorization: Token map: name: name model: platform.name ip: primary_ip4.address group: role.name insecure: true secure: false hosts_location: results pagination: true pagination_key_name: 'next'

This was working perfectly until I got to over 50 nodes. I then turned pagination on so that oxidized would make additional requests to netbox for the 2nd 50 nodes and so forth.

On the oxidized web interface, the first 50 nodes look just fine, but when I go to page 2, all of the nodes on page two are duplicated. I have a total of 56 nodes now defined in Netbox, but oxidized says there are 62 nodes. That's because 51 through 56 are there twice.

ianbarrere commented 4 months ago

This doesn't address the actual problem, but it may be a suitable workaround for you. You can provide a "limit" parameter in the Netbox URL to give you back more per page to avoid pagination completely. The URL I use in my source is this:

https://netbox.my.env/api/dcim/devices/?format=json&limit=1000&tag=oxidized

stevesaner commented 4 months ago

That does provide a workaround. Thanks for the suggestion.