During testing, we've seen some high memory usage in the library when it is applied to models that have a large number of object records stored in the database.
This has been traced to the model.objects.all().anonymise() call in anonymise_db which uses a queryset but appears to cache a significant amount of query metadata before anonymisation of the first object takes place.
Since we have once-only usage semantics for the model objects in the anonymise_db use case, we could use a Django queryset iterator to reduce memory consumption.
During testing, we've seen some high memory usage in the library when it is applied to models that have a large number of object records stored in the database.
This has been traced to the
model.objects.all().anonymise()call inanonymise_dbwhich uses a queryset but appears to cache a significant amount of query metadata before anonymisation of the first object takes place.Since we have once-only usage semantics for the model objects in the
anonymise_dbuse case, we could use a Django queryset iterator to reduce memory consumption.