netboxlabs / diode

Diode data ingestion for NetBox, from NetBox Labs
https://netboxlabs.com
Other
37 stars 1 forks source link

feat: OBS-398 - netbox-diode-plugin: create api endpoint for applying change sets #49

Closed Julio-Oliveira-Encora closed 8 months ago

Julio-Oliveira-Encora commented 8 months ago

Add an endpoint to apply changes (create and update) objects in the Netbox.

linear[bot] commented 8 months ago
OBS-398 Create API endpoint for applying change sets

In the NetBox diode plugin, we need to create an API endpoint for change sets to process by the plugin. `POST /api/plugins/netbox_diode_plugin/apply-change-set` Diode's reconciler service will make change sets based on ingested data and existing matching NetBox object states and send a request to this API endpoint. Payload will contain a list of changes to action with following fields: * `change_id` - \[required\] string containing change ID generated by diode backend during data ingestion * `change_type` - \[required\] string containing operation type: `create`, `update` * `object_type` - \[required\] string containing NetBox object type, i.e. `device` * `object_id` - integer containing NetBox object ID for change types other than `create` (i.e. we need object ID to update or delete \[in the future\] existing objects) * `object_version` - integer containing NetBox object's `objectchange` ID (see [https://docs.netbox.dev/en/stable/features/change-logging/](https://docs.netbox.dev/en/stable/features/change-logging/)), similarly to `object_id` required for change types other than `create` * `data` - \[required\] an object/dict containing properties corresponding to `object_type` NetBox model Example payload: ``` { "change_set_id": "", "change_set": [ {"change_id": "", "change_type": "create", "object_version": null, "object_type": "dcim.device", "object_id": null, "data": {"name": "router01", "device_type": "ISR4321", "device_role": "undefined", "site": "undefined", "platform": "IOS 15.6", "status": "Active", ...}}, {"change_id": "", "change_type": "update", "object_version": 1, "object_type": "dcim.device", "object_id": 123, "data": {"name": "router01", "primary_ipv4": "mgmt", ...}}, {"change_id": "", "change_type": "create", "object_version": null, "object_type": "dcim.site", "object_id": null, "data": {"name": "undefined", ...}}, {"change_id": "", "change_type": "create", "object_version": null, "object_type": "dcim.device_role", "object_id": null, "data": {"name": "undefined", ...}}, {"change_id": "", "change_type": "create", "object_version": null, "object_type": "dcim.platform", "object_id": null, "data": {"name": "IOS 15.6", ...}}, {"change_id": "", "change_type": "create", "object_version": null, "object_type": "dcim.manufacturer", "object_id": null, "data": {"name": "Cisco", ...}}, {"change_id": "", "change_type": "create", "object_version": null, "object_type": "dcim.device_type", "object_id": null, "data": {"model": "ISR4321", "manufacturer": "Cisco", ...}} ] } ``` Once we receive the request, we should action given change sets based on the `change_type`, i.e: ``` {"change_id": "", "change_type": "create", "object_version": null, "object_type": "dcim.device", "object_id": null, "data": {"name": "router01", "device_type": "ISR4321", "device_role": "undefined", "site": "undefined", "platform": "IOS 15.6", "status": "Active", ...}} ``` should create a new `Device` NetBox object using provided data. We should use atomic transactions and attempt to commit all changes in a set or rollback on any failure. Example use of Django's built-in `transaction.atomic` functionality ([docs](https://docs.djangoproject.com/en/5.0/topics/db/transactions/#django.db.transaction.atomic)): * [https://github.com/netbox-community/netbox/blob/develop/netbox/dcim/views.py#L115-L151](https://github.com/netbox-community/netbox/blob/develop/netbox/dcim/views.py#L115-L151) These change sets should be applied utilising existing corresponding NetBox models. Note that each NetBox object type (model) might have some requirements, i.e. `Device` requires `Site`, `DeviceTypeID` and `RoleID` which we should create first as a placeholders prior to creation of `Device` object itself. It's not specified yet, but might provide these placeholders to create in the change set's data while being generated by reconciler service (TBC). API endpoint should respond with with a success response or return an error. Success: ``` {"change_set_id": "", "result": "success"} ``` Error: ``` {"change_set_id": "", "result": "failed", "errors": ["failed to create device..."]} ``` (response structure may change as we find it appropriate during development and testing). This API should be unit tested appropriately.