Sort by relevance to current tags in prompt

[the-saif-ahmad]

I wonder if it would be possible to detect booru tags already in the prompt and have matches for the current token be sorted by a "most used with current tags in prompt on X, Y, Z booru sites" metric? And have it as a toggle option in the settings?

[DominikDoom]

The detection part is possible since I already use something similar to detect and display the tags in the live translation preview feature.

The main issue would be the metric itself, I have no local data about such "most used" or similar statistics. It would have to be an API request through danbooru for each tag. An API endpoint for related tags exists which provides exactly that, but I'm unsure what the usage limits are for that. Since it's a third party API it would definitely be optional though, maybe even require a personal API key the user has to create and input themselves depending on rate limits etc.

Another approach would be to analyze usage statistics over all generations and build a database of the user's personal tags over time, but that is also much less simple. Something similar was suggested in #165, which I closed at the time for being too out of scope for me to implement quickly. But I have kept it in mind as a possible future roadmap point.

[the-saif-ahmad]

I was thinking something more along the lines of a one-time calculation, just like with the tag list itself. Do a scrape of a major booru site, calculate the relevance between each tag. Do that for multiple boorus and store in a big json (or something more elegant) to ship with the repo? Then have an option to enable and which booru site to base relevancy on.

[DominikDoom]

For danbooru at least, a one time scrape would be possible since they provide the data in the API without needing to log in, but I am unsure how much space it would take and if it would be feasible to ship with the repo.

Just as an example of the response :

(await (await fetch("https://danbooru.donmai.us/related_tag.json?query=1girl&order=frequency")).json()).related_tags.map((x) => x.tag.name)

Running just that in the browser console in a new tab will already return the 100 related tags for "1girl" in a directly useable list, so getting the data isn't an issue. Just the amount of data it would entail.

The current tag list comes from a simple dump of the top 100k tags by post count, even when limited further that would be a huge amount of additional data if I were to get related tags for each of those. Especially if including all 100 related tags, but even with 10 it gets big fast. It's a similar issue to tag aliases, but those were much fewer in number and most tags don't have any, so they don't add as much overall. And I don't want to just arbitrarily pick a cutoff point after like 5k tags either, since quite a few tags that might be relevant when writing a prompt actually don't have that many posts on danbooru, they just describe a concept that is already well understood by most SD models.

I guess it could work when reusing data and just referencing by ID like a proper database would to prevent duplicates. But that is pretty ugly to do in json or CSVs, and I don't really want to deal with accessing an actual sqlite store from browser javascript through WebAssembly or something similar. So a static reference is definitely tricky (but still possible).

[the-saif-ahmad]

Yeah I figured space would be the immediate constraint. I definitely wouldn't mind if it was a separate download to be used as an extension, especially if there were versions with different cutoffs, or people could source their own lists. I think the convenience is worth the space tradeoff for me, but I agree an actual implementation requires a bit more thought.