albertyw
commented
4 months ago If you call all() on a table with many rows, it will retrieve all the data at once
I don't think that's true. It will retrieve data from the data if it is evaluated (e.g. through print() or otherwise). In this case, .all().delete() will issue a DELETE FORM <TABLE> without actually retrieving data so there should be no OOM in your python application.
Your pull request seems to tackle a different problem - since django-silk may be continually profiling and writing new rows into the database, it may conflict with DELETE operation unless the rows to be deleted are separate from newly written rows, hence the deadlock issues discussed in #265 and #294.
moonsikpark
sergei-iurchenko
https://github.com/jazzband/django-silk/blob/1cb46236b85f2f0b83296a02b64e9fc5f9021247/silk/models.py#L167-L170
If you call all() on a table with many rows, it will retrieve all the data at once, which will likely result in out-of-memory (OOM) errors.
I previously used Celery to call this function to avoid deadlock, as suggested in the referenced comment. However, due to OOM errors, our scheduler failed to clear the database, leading to excessive database usage.
I recommend setting a limit and deleting the rows recursively instead. I will submit a pull request to address this issue.