search

TheSpaghettiDetective / obico-server

Obico is a community-built, open-source smart 3D printing platform used by makers, enthusiasts, and tinkerers around the world.
https://obico.io
GNU Affero General Public License v3.0
1.44k stars 292 forks source link

[BUG/FEATURE] Deleting print history doesn't delete the underlying jpg (snapshot) and mp4 (timelapse) files #840

Open puterboy opened 8 months ago

puterboy commented 8 months ago

When you delete elements from the print history, it only changes the entries in the database (db.sqlite3), marking them as 'deleted' but it does NOT delete the actual snapshot and video timelapse storage.

  1. First, this notion of deleting is not very effective from a storage perspective in that it only superficially cleans up the print history display list but it leaves all the now zombie history elements in storage so it has no effect on disk usage.
  2. Second, this behavior is non-obvious as one would typically expect a 'delete' function to also delete the underlying storage and not just the metadata stored in db.sqlite3.

My understanding is that the cloud-based, hosted version "solves" the accumulating storage problem by just blanket deleting the any snapshots and videos older than 6 months, but:

  1. This requires a separate and potentially manually process to "expire" old storage on some regular basis
  2. This "blanket" deletion process also deletes the snapshots and videos of non-deleted print history elements which presumably you are purposely saving by dint of the fact that you didn't delete them from the print history! So, you end up having elements in your print history that no longer have working snapshots or timelapse videos associated with them

I wrote some klugey bash scripts that you can manually use to delete the snapshots and videos that are marked as deleted in db.sqlite3 -- see: https://github.com/TheSpaghettiDetective/obico-server/issues/762

However, it would seem that the right thing to do would be to include file deletion when 'delete' is selected from the UI. Any reason not to do so?

kennethjiang commented 7 months ago

This has been brought up a few times. The problem with deleting the underlying files while the print itself (and the printer) is deleted is file operations are usually expensive (both in terms of time to execute and $ amount). So it's a common practice for the web application to defer this kind of tasks to dev ops.

We are relying GCP storage lifecycle management to do things like this efficiently. You are welcome to come up a mechanism for self-hosted server, as long as it's compatible with the way Obico cloud works.