[RFC] Checking For Updates to Scene StashID Matches

[mmenanno]

Problem

Once a scene has been matched, a user typically wouldn't be able to receive any updates if details have been updated (maybe males were added or titles were fixed), or if the scene was deleted as a duplicate. There is an update function for performers currently but I think it could be improved as well along with adding the scene behaviour, though this issue will focus on the scene side of things as the performer behaviour is likely similar if not the same and could later be extended.

I believe the solution for this lives on both Stash and Stash-Box sides.

Stash Backend Side

Currently a StashID in Stash is made up of an endpoint and a stash_id. I believe another attribute should be added to it, to store the lasted updated_at time of the external resource. Not hard set on the name but this field could be something like external_updated_at.

Stash-Box Backend Side

For the sake of making the process optimal Stash-Box should have a new query findUpdatedScenes that takes an array of SceneUpdateInput structured something like:

{
  "scene_id": "stash_id",
  "last_recorded_update": "external_updated_at"
}

This call can return a updated Boolean, deleted Boolean, and Scene object for each input.

• If the last_recorded_update is NOT older than the updated_at on the scene_id, it will return updated: false , deleted: false, and no scene object
• If the last_recorded_update is older than the updated_at on the scene_id but the scene has been deleted, updated: true, deleted: true, and no scene object
• If the last_recorded_update is older than the updated_at on the scene_id and the scene is not deleted, updated: true, deleted: false, and the updated Scene object

Stash Frontend Side

The scene tagger view will get a new Batch Check for Updates button. This will collect all the scenes the user wants to check (whether this is all scenes or just a sub-set) and call back to Stash-Box with findUpdatedScenes.

• If the scene has no updates the scene item will show a message of No new updates
• If the scene has been deleted the scene item will show a message of Scene has been deleted with an external link to the ID so the user can investigate
• If the scene has been updated the scene item will show the returned scene object like it normally would on a scrape (it would be nice if we could highlight updates in a different colour, but that would be scope of another issue, since that would also generally improve any scraping, not just updates).

Buttons could also be added to Show only updated and Show only deleted for quick filtering of the list.

[stg-annon]

I'm wondering why we wouldn't do date checking on the stashapp side of things? In that situation we would just need get the updated value for each scene which can currently be done with findScene and wouldn't require any changes to Stash-Box, something for bulk querying scenes like findScenes might be helpful but not necessary to accomplish this.

[MikeLund]

An additional solution could even be something like an RSS feed from stashbox, providing a list of StashIDs that have been recently changed? Stashapp could query this automatically every once in a while, compare the results to its own StashIDs, then it knows what scenes should be queued for an update.

(On an unrelated tangent, even a similar thing with oshash would seem smart? I don't know much about how stashdb's backend is working, but it'd feel like RSS-esque requests would be the most sustainable longterm solution anyways? It feels "wrong" to send hundreds of thousands of graphql requests to stashbox when it's unlikely the majority are going to give any positive responses...)

[WithoutPants]

I think this can be broken up into smaller component parts:

The following would be an initial mvp:

• add the updated_at field to the stash id entities, have them populated, and add functionality to check, display and update scene data for tagged scenes

The other parts are more of an optimisation of this process.