.fmwk does not count scrobbles with redirected artist names

[rzumer]

When a user disable automatic "corrections" (redirects) on their scrobbles, fmwk cannot pick up scrobbles with artist names that are normally redirected.

Example: 小椋 佳 redirects to 小椋佳. I have corrections disabled and scrobbled an album credited to 小椋 佳. fmwk detects 0 scrobbles (even after an fmupdate) using .fmwk with no parameters immediately after scrobbling, and when writing the name manually.

If it's possible to combine unredirected and redirected alias results at least when using .fmwk with no parameters, that would be much better.

[rzumer]

Given that:

• Only combining scrobbles when querying an uncorrected alias would be confusing to users;
• Combining more than two aliases' scrobbles would not be possible that way; and
• The last.fm API does not seem to have a reverse lookup for redirected artist names

Here is my suggested implementation:

• Cache aliases in an internal database
• Refresh cache for an artist when it is queried (i.e. check for a correction on fmwk query and update the internal cache)
• Combine scrobbles of all cached aliases of an artist when counting them

This requires users to look up each alias of an artist for all their scrobbles to be counted, but is consistent, works globally and, should be fairly lightweight.

It would also make it easy to provide functions for administrators to set up their own aliases for alternative spellings not handled by last.fm.

[th0mk]

Thank you for the detailed issue. We already store aliases in the database and they're indeed retrieved when someone uses whoknows and it gets corrected.

When I look at the alias you provided I can see we already store it in our alias table:

And the artist in our artist table:

Our locally stored data (top artists/albums/tracks for every user) for users also runs through these cached aliases. This is because we use their recent scrobbles for updating that data and Last.fm sometimes gives us uncorrected names in there, even when a user has corrections enabled (see this issue I filed). This is what makes supporting non-corrected artists difficult.

This means:

• We don't store any whoknows records for the non-corrected artist (小椋 佳) so there are no other users that have this artist stored in our database, no matter if they have corrections enabled or not.

• If users wanted to disable corrections for their scrobbles, this would also have to be an option for them to set in the bot since we always correct recent scrobbles. The Last.fm API doesn't allow us to see what a user has picked.

• This would then also be an option when using whoknows, but you'd only see other users that also have corrections disabled in the bot.

Is this a solution that you would like to see and/or do you have any additional feedback?

[rzumer]

Just to make sure we are on the same page- I'd like unredirected scrobbles for, say, 小椋 佳, to be counted when running a who-knows command on either 小椋 佳 or 小椋佳. Currently 小椋 佳 redirects to 小椋佳, and only scrobbles for 小椋佳 are counted. Redirecting artist lookups in the bot is fine in my opinion, I would just like scrobbles for its aliases to be counted, so that people can see that I have listened to an artist even if my scrobbles point to an alias.

Since you are already storing a list of aliases, wouldn't it be possible to count scrobbles for all (known, stored) aliases of a queried artist, then sum them up, when running the who-knows command?

Please let me know if I missed something, maybe there is an API-level constraint I am not aware of.

[th0mk]

The initial data we get for users is based on their top 4000 artists (this is directly from Last.fm). After that we update this data with your recent scrobbles. You can read more about our initial index/update process here (recommend reading that before reading this reply further)

If you have corrections enabled on the Last.fm site at the time of getting your initial data (indexing), the count we have in the bot for 小椋佳 should be the same as 小椋 佳 since we use the corrected artist name that is Last.fm returns to us to look in our database.

If you have corrections disabled on the Last.fm site at the time of getting your initial data (indexing), the count we have in the bot for 小椋佳 will still be based on whatever count Last.fm gives us for 小椋佳. If we update you later (which is based on recent scrobbles) the count will almost certainly differ from Last.fm, since we correct the artist in your recent scrobbles but Last.fm doesn't.

Long story short: We see Last.fm as our 'source of truth' and try to replicate their statistics as close as possible. We always correct artist names when we update users using their recent plays, so we currently don't support disabling redirections. What you're asking for is already implemented. We don't have any plays stored for 小椋 佳, only for 小椋佳.

[rzumer]

What you're asking for is already implemented. We don't have any plays stored for 小椋 佳, only for 小椋佳.

I edited my scrobbles for this particular artist since I brought up that example, in case you are basing that on the database contents. But my scrobbles for artist aliases are definitely not counted in the who-knows command, regardless of recency. I can point to Sadistic Mika Band as well, which redirects to サディステイック・ミカ・バンド. I have a few scrobbles in each and only the ones associated with the "correct" spelling are ever counted.

If you have corrections disabled on the Last.fm site at the time of getting your initial data (indexing), the count we have in the bot for 小椋佳 will still be based on whatever count Last.fm gives us for 小椋佳.

Does that mean scrobbles for unredirected aliases will always be "missing" from the bot's count if they are sent before initial indexing? Isn't there a way to retrieve scrobbles for the unredirected alias and "merge" them internally?

Again I think it's fine to follow last.fm redirects, I'd just like unredirected scrobbles to be counted too, so my Sadistic Mika Band scrobbles would be added to my サディステイック・ミカ・バンド ones when calling fmwhoknows.

[th0mk]

Does that mean scrobbles for unredirected aliases will always be "missing" from the bot's count if they are sent before initial indexing?

Yes. If we would base our data purely on all of your scrobbles and not on the provided top artists/albums/track calls we would have to do tons of API calls, especially for users with a lot of scrobbles. Every 1000 scrobbles is 1 API call, and we have to be strict on how much API calls we do.

Again I think it's fine to follow last.fm redirects, I'd just like unredirected scrobbles to be counted too, so my Sadistic Mika Band scrobbles would be added to my サディステイック・ミカ・バンド ones when calling fmwhoknows.

I have just tested to make sure this works and if you scrobble this and then run .fmupdate it will be properly counted for サディステイック・ミカ・バンド. Are you sure this happens on new scrobbles?

[rzumer]

Yes, I also just tested it today, with 荒井由実 (redirected to 松任谷由実). I scrobbled several tracks, ran an update in the middle and then .fmwk multiple times (parameterless while playing, and with both artist name variants specified). It looks up 松任谷由実 and does not detect any scrobbled track for me.

[th0mk]

.fmwk gets the playcount for an artist directly from Last.fm in some cases (more specifically when it cant find an artist directly in the database or you've not been updated recently). It used to always get the playcount from Last.fm, but because we introduced the crowns game we temporarily use the playcount in our database sometimes to prevent hammering the Last.fm API.

Other commands (whoknows album, whoknows track, artist etc) still directly get their playcount from Last.fm for the user who requested the command and it will probably stay this way, since its more up to date and returned in the API call to Last.fm anyway.

If we would start counting plays differently then Last.fm on purpose and use this in more commands it would become a slippery slope.

[rzumer]

I understand your perspective, for me summing up scrobbles for aliases for fmwk alone would be plenty since that's what matters for the crown game and it's the reason I opened this issue in the first place, but that would be inconsistent with how other commands work if it's the only one to implement that behavior. If I find some free time I might try and implement it myself on a fork and see how it performs.