esmero / ami

Archipelago Multi Importer. A module of mass ingest made for the masses
GNU Affero General Public License v3.0
2 stars 4 forks source link

Create a Log Entity for AMI operations or CSV report? #92

Closed DiegoPino closed 1 month ago

DiegoPino commented 2 years ago

What?

I'm not sure yet but I feel this might be a needed discussion related to #37. AMI batch sets are hard to debug without proper logging. And even if we can generate drupal dblog messages those are not very safe/reliable if we need to filter/find the ones we need or might not even stay around long enough. Also sometimes a single AMI set generates multiple iterative operations.

My only concern on going the custom ENTITY for logging route is performance. A Single AMI set can have 10K entries. This would generate 10K log entities number of operations number of AMI sets... do we want this? And how long do we want to keep logs?

But one benefit of this approach is also undo operations and revert operations. But also allow to retraced lost entities. But then we have the data always on.

The second option is to write logs/reports into CSVs. Each operation generates a CSV that can be downloaded IF the user decides so, if not gets lots in the Universe of lost socks and CSVs, or can be all be attached in a temporal manner to the AMI set for future reference (a CSV would be tiny in comparison but would still be permanent). The bad thing about the CSV is the lack of proper filtering/querying that a DB/entity would give us (via the UI, you can still use Excel/GoogleSheets for that). In a custom Entity I could filter by date/type of operations, who did it, etc.

Anyone has comments/suggestions?

DiegoPino commented 1 month ago

Deprecated. We went for a log file