search

nautobot / nautobot-app-ssot

Single Source of Truth for Nautobot
https://docs.nautobot.com/projects/ssot/en/latest/
Other
37 stars 38 forks source link

Update Contrib to use bulk CRUD operations for Django database models #482

Open Renrut5 opened 3 months ago

Renrut5 commented 3 months ago

Environment

The contrib code currently performs CRUD operations one at a time. This can take a very long time in instances where the operations get into the thousands. Part of the reason

Proposed Functionality

Take advantage of Django's bulk CRUD methods (bulk_update(), bulk_create(), etc.) in SSoT Contrib.

Use Case

Use case includes any SSoT job syncing data into Nautobot in order to improve time required to create many objects.

mzbroch commented 3 months ago

This is a good performance improvement idea. Bulk operations have different ways of validating / verifying during clean up - let's ensure this is not missed!

Renrut5 commented 3 months ago

Have been discussing with @jdrew82 a bit on this and we both agree the changes would need stay within the SSoT app and not impact the diffsync library.

Kircheneer commented 3 months ago

This is something I think would be useful but would definitely have to be configurable as I wouldn't like for it to be the default behaviour.

jdrew82 commented 3 months ago

This is something I think would be useful but would definitely have to be configurable as I wouldn't like for it to be the default behaviour.

Absolutely should NOT be the default behavior to use bulk. That should only be used if designated by the dev.