A rework of the torrent creation workflow: previously a Torrent record was always created with a corresponding TorrentContent record, which would usually be empty; following this a classify_torrent queue job would attempt classification then update the TorrentContent record.
Following this update, Torrent records will always be created in isolation, and a process_torrent job will then run in the queue. This will not only classify the torrent, but also perform any other tasks like search reindexing. For torrents that have already been matched to a piece of content, rematching will not occur (unless specified in the CLI command, see below), which saves a significant amount of work.
A new entity type, TorrentHint has been created for providing hints to the classifier (currently used only by the import tool). Previously any hints for the classifier were added directly to the TorrentContent record, which was a conflation of 2 different things (the classification result, and hints for the classifier), which is problematic when it comes to reclassification.
Additionally, a new CLI command, reprocess has been added, which will reprocess all torrents, classify them, and update the search index. For already matched torrents, rematching will only occur when passing the --rematch flag.
A few reasons for this change:
It will prevent unclassified (but classifiable) torrents showing at the top of the list in the WebUI, which is confusing
The new search index will require reindexing of all torrents, and the CLI command provides a simple way to do this
In future, we'll want to hang further steps off the process_torrent job, such as rules-based deletion, and this provides the groundwork for that
A rework of the torrent creation workflow: previously a
Torrentrecord was always created with a correspondingTorrentContentrecord, which would usually be empty; following this aclassify_torrentqueue job would attempt classification then update theTorrentContentrecord.Following this update,
Torrentrecords will always be created in isolation, and aprocess_torrentjob will then run in the queue. This will not only classify the torrent, but also perform any other tasks like search reindexing. For torrents that have already been matched to a piece of content, rematching will not occur (unless specified in the CLI command, see below), which saves a significant amount of work.A new entity type,
TorrentHinthas been created for providing hints to the classifier (currently used only by the import tool). Previously any hints for the classifier were added directly to theTorrentContentrecord, which was a conflation of 2 different things (the classification result, and hints for the classifier), which is problematic when it comes to reclassification.Additionally, a new CLI command,
reprocesshas been added, which will reprocess all torrents, classify them, and update the search index. For already matched torrents, rematching will only occur when passing the--rematchflag.A few reasons for this change:
process_torrentjob, such as rules-based deletion, and this provides the groundwork for that