search

tibiamaps / tibia-kill-stats

4 stars 3 forks source link

Kill statistics bugs/suggestions to report to CipSoft #2

Open mathiasbynens opened 2 years ago

mathiasbynens commented 2 years ago

Add missing creatures/bosses

Lord Retro (or Retros Treasure) is missing from the kill stats. (Reported as Ticket#20221101420075.)

Same with King Chuck / Orger Treasure and The Percht Queen.

Exclude summons

Summons are currently counted, but should not be. Including summon kills is problematic because, for example, it makes the stats for Mindmasher useless since he summons copies of himself. (It's also probably why there have been 146 global poodle kills since we started collecting this data, despite no known poodle spawn existing in the game.)

Apply consistent pluralization/casing to creatures and boss names

The general approach of the kill stats seems to be:

  1. Creature names are in lowercase and pluralized (e.g. rats instead of rat).
  2. Boss names are in uppercase and should not be pluralized (e.g. Bloodback and not Bloodbacks).

These rules make sense! The problem is there are currently some exceptions that need fixing. For example, the Diseased brothers' names are currently pluralized but should not be. Here's the list of (what I believe to be) incorrectly pluralized names, based on the fact these are bosses or boss-like triggerable spawn creatures:

These should all be changed to remove the s, in order to be consistent with other such bosses/creatures.

And then as @LeeKunTibia points out, there are also some bosses or boss-like creatures with incorrectly lowercased names:

Finally, there are also creatures that are incorrectly capitalized:

Update: The iks creature names have been fixed by lowercasing them in January 2023.

Fix typos in creature names

Align kill stats updates with server saves

Request: Please update the kill stats at server save instead of at 04:00 in the morning. This would ensure every kill stats update aligns with an entire "server day" (in between two server saves), which would improve the usefulness & accuracy of the stats. Currently, if a rare creature is listed as being killed 2x in the kill stats, there is no way to tell whether those two kills happened in a single server day (between 10:00 and 09:59), or across two server saves (one before 04:00 and one after).

Reality reaver vs. reality reavers

There are two distinct entries for reality reaver and reality reavers, with the latter being killed much more commonly: https://github.com/tibiamaps/tibia-kill-stats/blob/4fbe986aefbcb4ae1023ea8c7ada9664f989ff0e/data/_global-total/kills.json#L1334

"reality reaver": 197832,
"reality reavers": 25994723,

Should these be merged? https://github.com/tibiamaps/tibia-kill-stats/blob/4fbe986aefbcb4ae1023ea8c7ada9664f989ff0e/data/zuna/2022-11-12.json#L3484-L3497

Double-counted kills

As reported by @muriloricci, The Enraged Thorn Knight seems to get counted twice for every completed fight against him. Both the first form where he's on top of his horse, and the final form seem to be counted as separate kills since the creature has the same name in both forms.

(There's no way people kill the earth portal boss more often than Lloyd.)

The same goes for The Sandking:

It seems The Sandking numbers are also doubled as you kill two of them during the boss fight. One in the 3-boss split phase and the other one in the last phase when it walks slowly.

Lack of granularity for horses, butterflies, and nomads

The kill stats do not currently distinguish between:

It would be useful to have separate entries for these since they appear separately in the Bestiary.

LeeKunTibia commented 2 years ago

Some bosses don't have their names capitalized, which makes them appear among creatures:

mathiasbynens commented 2 years ago

I’ve added some code that checks if every creature/boss in a custom category is actually found in the kill stats. This revealed some missing entries which I’ve corrected in 4221c5f13831fb41e15bd45c111999c7fdc0fde2, which matches the current kill stats spelling.

npm run analyze currently warns about the following missing entries:

Category nemesis-boss: expected 105 entries but got only 101. Missing entries: Set(4) {
  'Bane Lord',
  'King Chuck',
  'The First Dragon',
  'The Percht Queen'
}
Category archfoe-boss: expected 99 entries but got only 94. Missing entries: Set(5) {
  'Gelidrazah the Frozen',
  'Kalyassa',
  'Lord Retro',
  'Tazhadur',
  'Zorvorax'
}
Category boss: expected 272 entries but got only 263. Missing entries: Set(9) {
  'Bane Lord',
  'King Chuck',
  'The First Dragon',
  'The Percht Queen',
  'Gelidrazah the Frozen',
  'Kalyassa',
  'Lord Retro',
  'Tazhadur',
  'Zorvorax'
}

These are either not yet in the historical kill stats (since we only started collecting them back in August 2022), or just creatures/bosses that don’t appear to be listed in the kill stats at all (Lord Retro, King Chuck).

mathiasbynens commented 2 years ago

We now maintain a complete mapping from names as they appear in the kill stats to the "pretty", proper names: https://github.com/tibiamaps/tibia-kill-stats/blob/main/normalize-names.mjs This makes it significantly easier to define creature or boss categories, since we don't have to use the pluralized and often weirdly cased forms everywhere.

mathiasbynens commented 2 years ago

~TODO: add oddities from https://github.com/tibiamaps/tibia-kill-stats/commit/2f2976280bd6902b9a7c7a1494d295f7bcb90936~ Done.

muriloricci commented 1 year ago

"The sorting of the Iks creatures is now correctly listed in the kill statistics", however the word "Iks" is still captalised.

https://www.tibia.com/news/?subtopic=newsarchive&id=7112

mathiasbynens commented 1 year ago

"The sorting of the Iks creatures is now correctly listed in the kill statistics", however the word "Iks" is still captalised.

https://www.tibia.com/news/?subtopic=newsarchive&id=7112

Thanks for the heads-up!

I’ve taken a quick look and I think this change doesn’t affect this repository. Here’s the before/after comparison for Vunira, for example:

In other words, I think the JSON we get from the TibiaData API is already sorted independently of how the kill stats pages are sorted. So we should be good 👍

Let me know if I got any of that wrong!

mathiasbynens commented 1 year ago

The Percht Queen is also missing from the kill stats. (Added to the top post.)

muriloricci commented 1 year ago

"Adjusted capitalization of Iksupan creatures to correct display within kill statistics."

https://www.tibia.com/news/?subtopic=newsarchive&id=7135

mathiasbynens commented 1 year ago

"Adjusted capitalization of Iksupan creatures to correct display within kill statistics."

https://www.tibia.com/news/?subtopic=newsarchive&id=7135

Any idea what this means? I don't see any difference on https://www.tibia.com/community/?subtopic=killstatistics&world=Vunira.

LeeKunTibia commented 1 year ago

"Adjusted capitalization of Iksupan creatures to correct display within kill statistics." https://www.tibia.com/news/?subtopic=newsarchive&id=7135

Any idea what this means? I don't see any difference on https://www.tibia.com/community/?subtopic=killstatistics&world=Vunira.

Maybe it won't update until the next Kill Stats update in ~9 hours.

muriloricci commented 1 year ago

"Adjusted capitalization of Iksupan creatures to correct display within kill statistics." https://www.tibia.com/news/?subtopic=newsarchive&id=7135

Any idea what this means? I don't see any difference on https://www.tibia.com/community/?subtopic=killstatistics&world=Vunira.

Maybe it won't update until the next Kill Stats update in ~9 hours.

It seems we now have both "Iks" and "iks" in the kill statistics. :P

muriloricci commented 1 year ago

It seems the bosses in the 25th Tibia Anniversary also are not displayed in the kill statistics. This probably doesn't matter too much, but due to a bug some worlds were able to kill them yesterday so I'm highlighting that here.

https://www.tibia.com/forum/?action=thread&threadid=4938297&postid=39363519#post39363519

mathiasbynens commented 1 year ago

"Adjusted capitalization of Iksupan creatures to correct display within kill statistics." https://www.tibia.com/news/?subtopic=newsarchive&id=7135

Any idea what this means? I don't see any difference on https://www.tibia.com/community/?subtopic=killstatistics&world=Vunira.

Maybe it won't update until the next Kill Stats update in ~9 hours.

It seems we now have both "Iks" and "iks" in the kill statistics. :P

I'll wait a week until we no longer have both in the stats. At that point, I'll write a script to retroactively rename the creatures throughout this repository. For the 7 days where entries contain both Iks and iks names, I'll have to merge the entries and sum up the kills.

Update: That script is now here: https://github.com/tibiamaps/tibia-kill-stats/blob/main/merge-entries.mjs

Update 2023-01-18: Done.

mathiasbynens commented 1 year ago

~TODO: add oddities from 5cca6336d081986cffb8420ce4be7a49b0d378fc~ Done.

muriloricci commented 1 year ago

It seems The Sandking numbers are also doubled as you kill two of them during the boss fight. One in the 3-boss split phase and the other one in the last phase when it walks slowly.

mathiasbynens commented 1 year ago

It seems The Sandking numbers are also doubled as you kill two of them during the boss fight. One in the 3-boss split phase and the other one in the last phase when it walks slowly.

Thanks! Added to the top post.

muriloricci commented 1 year ago

I checked the data available now to see if I could statistically find bosses that tend to be killed by an even number. I believe on average we should have balanced numbers where a boss was killed by an amount equals to an even or an odd number, so anything unbalanced could be suspicious.

So here are the top bosses in the list (number on the left is the amount of files that the boss was killed by a number that is even, number on the right is the amount of file that the boss was killed):

Dragon Essences: 927/981 (94.50%) The End Of Days: 10074/11249 (89.55%) The Enraged Thorn Knight: 6824/11045 (61.78%) The Sandking: 5128/9015 (56.88%) Gelidrazah the Frozen: 575/1105 (52.04%) Black Vixen: 6766/13426 (50.39%) Timira the Many-Headed: 6752/13406 (50.37%) Utua Stone Sting: 6739/13420 (50.22%) Grand Master Oberon: 6724/13394 (50.20%) Yirkas Blue Scales: 6731/13421 (50.15%) Scarlett Etzel: 6695/13395 (49.98%) Katex Blood Tongue: 6704/13420 (49.96%) Kalyassa: 552/1105 (49.95%) Shadowpelt: 6701/13426 (49.91%) Bloodback: 6701/13427 (49.91%) Jailer: 6667/13361 (49.90%) Sharpclaw: 6680/13427 (49.75%) Srezz Yellow Eyes: 6668/13420 (49.69%) Grand Canon Dominus: 6608/13389 (49.35%) Drume: 6592/13357 (49.35%) Preceptor Lazare: 6603/13383 (49.34%) Darkfang: 6607/13427 (49.21%)

I believe this doesn't tell us anything new really but kind of confirms that there's something off with The Sandking and The Enraged Thorn Knight, however I expected their percentage to be higher than that.

mathiasbynens commented 1 year ago

Interesting findings!

muriloricci commented 1 year ago

Oops, Kikaro pointed out that my output was wrong. I've fixed it and here are the new values:

Feroxa: 389/389 (100.0%) Dragon Essences: 927/927 (100.0%) Goshnar's Greed: 2186/2186 (100.0%) Ferumbras Soul Splinter: 5170/5217 (99.1%) The End Of Days: 10074/10177 (98.99%) The Sandking: 5128/5211 (98.41%) Despair: 1762/1794 (98.22%) The Enraged Thorn Knight: 6824/7043 (96.89%) Bone Capsule: 1454/1836 (79.19%) The Duke Of The Depths: 1291/1989 (64.91%) The Baron From Below: 1155/1798 (64.24%) Rage of Mazoran: 3063/5398 (56.74%) Jaul: 502/887 (56.6%) Gelidrazah the Frozen: 575/1104 (52.08%) Frenzy: 2447/4711 (51.94%) Piñata Dragons: 165/318 (51.89%) Dragon Wardens: 477/932 (51.18%) The Count Of The Core: 3402/6650 (51.16%) The Mutated Pumpkin: 170/335 (50.75%) Fire Overlords: 123/243 (50.62%) Gaffir: 6224/12299 (50.61%) Black Vixen: 6766/13374 (50.59%) Timira the Many-Headed: 6752/13360 (50.54%) Utua Stone Sting: 6739/13383 (50.35%) Grand Master Oberon: 6724/13355 (50.35%) Yirkas Blue Scales: 6731/13385 (50.29%) Jailer: 6667/13265 (50.26%) Bloodback: 6701/13372 (50.11%) Shadowpelt: 6701/13375 (50.1%) Katex Blood Tongue: 6704/13381 (50.1%) Kalyassa: 552/1104 (50.0%) Charged Anomaly: 1/2 (50.0%)

kik-tibia commented 1 year ago

That matches my results now :)

Some other interesting things: The Sandking is commonly divisible by 4: 5127/5211 (98.39%), because you kill the 3 forms that share a life pool and then the final form. Shadow tentacle kills are divisible by 5: 7872/7872 (100.00%) with a 100% occurrence. 5 of them spawn at the start of the Lady Tenebris fight and never respawn. People are often efficient in making ebb and flow access, with an exact multiple of 20 hazardous phantoms happening this often: 1414/3709 (38.12%). 8 soul splinters spawn in the Feru final fight, and we can see how often people succeed in this stage 5142/5217 (98.56%).

LeeKunTibia commented 1 year ago

Very nice find guys, good job!

Edit: I'm really curious about this duplicated reality reavers entries. The stats also have stabilizing reality reavers, so that shouldn't be it. It also doesn't happen with the other Otherworld creatures... Are we missing something? Could there be more than one Reality Reaver ID? One that spawns somewhere else.

We should be able to "detect" it by paying attention to the number of kills in our Bestiary entries since different IDs should not add to the same creature.

kik-tibia commented 1 year ago

I've just taken a look at my Loot log, which I try to save regularly and upload to Tibia Wiki once a week. On Sunday I did a GT run, and there many Loot of a reality reaver entries, but I can see one instance of Loot of reality reaver. This line occurred after killing the GT bosses, and would have been a reality reaver that was killed on the way out from the hub.

LeeKunTibia commented 1 year ago

I've just taken a look at my Loot log, which I try to save regularly and upload to Tibia Wiki once a week. On Sunday I did a GT run, and there many Loot of a reality reaver entries, but I can see one instance of Loot of reality reaver. This line occurred after killing the GT bosses, and would have been a reality reaver that was killed on the way out from the hub.

The plot thickens! That's helpful, it points to the location where we could be able to find one of the special types of Reality Reavers. That is, assuming they can't all spawn at the same monster homes as varying spawns, just like Burster and Dreadful Disruptor.

Probably won't be useful, but for the record: the ID of the Bestiary Reality Reaver is 1224

LeeKunTibia commented 1 year ago

Turns out the reality reavers issue no more than a bug on CipSoft's side. The one without the a and that is killed fewer times are those at the Otherworld entrance, the weaker versions of the Otherworld creatures that don't count for bestiary etc.

image

There is simply a bug with this creature's name that it doesn't have an article, making it appear separetly in the Kill Stats and differently in the Loot log. I'll report that to CipSoft.

LeeKunTibia commented 1 year ago

I just learned after reading this post that the Goshnar's Megalomania has disappeared from the Kill Statistics after the bosstiary change. I have verified and confirmed that it's truth and it's missing. Ferumbras was just renamed as we know; can't tell about Devovorga until the event takes place.

It's already been reported but the data between the change and the fix will be missing.

muriloricci commented 1 year ago

I noticed last night that The Unwelcome doesn't appear in the statistics. Maybe one could justify that since players don't actually attack it but the boss dies nonetheless in the fight so that's odd to me.

Additionally, both The Unwelcome and Brother Worm are not captalised in the server logs:

22:03:33 Brother worm loses 1097 hitpoints due to an attack by Holy Lightbringer. 22:03:34 Using one of 1102 strong mana potions... 22:03:34 You gained 155 mana. 22:03:35 You lose 24 hitpoints due to an attack by the unwelcome. 22:03:35 Brother worm loses 1007 hitpoints due to an attack by Holy Lightbringer. 22:03:36 Using one of 1101 strong mana potions... 22:03:36 You gained 149 mana. 22:03:36 Brother worm loses 1142 hitpoints due to an attack by Holy Lightbringer. 22:03:36 Flaxoro loses 1092 hitpoints due to an attack by brother worm. 22:03:36 Lady van Mave loses 1092 mana due to an attack by brother worm.

Brother Worm is listed in the kill statistics as a boss (listed as "Brother Worm" instead of "brother worm")

muriloricci commented 1 year ago

CipSoft official reply on this issue up to this point:

Hello,

I'm contacting you to inform you about some feedback I received concerning your request concerning the kill statistics.

First about the bosses that are not in the kill statistics. I understand that you want the statistics to be as complete as possible. But there are reasons why certain bosses are not listed there. Unfortunately, I can't elaborate on the exact reasons but be assured that we evaluated the mentioned bosses carefully and will not be added to the statistics.

About creatures like horses, butterflies, and nomads using the same kill statistics. This was a deliberate decision in the past as the monsters have the same name. A splitting of the statistics into single values is not seen as necessary.

We also evaluated your request on synchronizing the kill statistic update with the server saves. Even though we understand the intention behind the request and think that there are valid reasons to do so, there are also more pressing reasons to keep the synchronization time as it is.  The reason for this decision is that updating the statistics loads quite heavily on the system. Therefore it was deliberately set at a time with little system load. As the highest system load is at the time of the server saves, adding the statistics update on top of it could cause problems. This means we will not make any changes to the present update time.

But we will address the naming inconsistencies of the reported bosses when it comes to pluralization, casing and typos.

And we will also take care of the double counts in the kill statistics for the reported bosses The Enraged Thorn Knight and The Sandking.

I know that my feedback to your requests is not all positive but we thoroughly evaluated all of them, and I hope you can at least understand our decisions.

But thank you very much for your time and efforts on this topic and for sharing your ideas with us. We appreciate that.

If you have any further concerns, please contact me again.
mathiasbynens commented 1 year ago

I'll count it as a win! Although not all issues will be fixed, it's nice that CipSoft is listening, and that they are at least willing to fix the pluralization + casing + double counting issues. Thank you for reporting back here!

mathiasbynens commented 1 year ago

Goshnar's Megalomania is now included in the website kill statistics.

https://www.tibia.com/news/?subtopic=newsarchive&id=7203

mathiasbynens commented 1 year ago

With today's server save, we have fixed the following issues: […] We have fixed an issue regarding wrong multiple counts of Boss kills for the kill statistics.

— https://www.tibia.com/news/?subtopic=newsarchive&id=7216

muriloricci commented 1 year ago

With today's server save, we have fixed the following issues: […] We have fixed an issue regarding wrong multiple counts of Boss kills for the kill statistics.

— https://www.tibia.com/news/?subtopic=newsarchive&id=7216

One of them is certainly The Thorn Knight since there was a change to it in the game files. I would assume the other one is The Sandking following details from the email I shared here, but let's check that later on.

EDIT: It's confirmed to be at least The Sandking and The Enraged Thorn Knight.

kik-tibia commented 1 year ago

Doctor Marrow is missing from the kill stats. It's reasonable to expect it to be always 0 for Killed by Players (he never actually dies, just disappears after losing some hp), but when he actually kills someone, you should see at least something in the kill statistics for that. Two people died to him yesterday on Nefera: https://www.tibia.com/community/?name=Katilak https://www.tibia.com/community/?name=Paladinmon But Doctor Marrow is nowhere to be seen in the kill statistics yesterday. This is true for all servers, there is no record of him.

muriloricci commented 1 year ago

Not kill statistics but it seems there's an extra space for "shard of Magnor" in the King Zelos fight.

16:05:38 You lose 245 hitpoints due to an attack by a shard of Magnor . 16:05:38 You lose 146 hitpoints due to an attack by a shard of Magnor . (10% damage reduction by Sap Strength)

mathiasbynens commented 1 year ago

The 2023 Summer Update introduced two new creatures that are incorrectly capitalized:

These are more like creatures than they are bosses. If CipSoft decides they are somehow more boss-like, then their names shouldn’t be capitalized.

I’ve updated the top post accordingly.

muriloricci commented 1 year ago

I just realised Raging Mage, despite being a boss, has an indefinite article displayed in the server log. Maybe when looking the boss you also see it there.

16:32:05 A Raging Mage loses 726 hitpoints due to an attack by Garthrater Adweznor. 16:32:05 A Raging Mage loses 851 hitpoints due to an attack by Toxic Smokie. 16:32:05 A Raging Mage loses 318 hitpoints due to an attack by Garthrater Adweznor. 16:32:05 A Raging Mage loses 1593 hitpoints due to an attack by Czin czi.

(And yeah, I missed this boss because people killed it too fast.)

muriloricci commented 10 months ago

As seen here, it is likely that Despor is counting twice in the kill statistics.

I've reported this already back in December 2023 (ticket #20231209420064) but I got no response from the support team yet.

EDIT: Got a first update!

Hello,

thank you for your report.

We are still looking into this issue, so at this point we can neither confirm nor refute your report. However, if we can confirm it, we will be sure to make the necessary adjustments.

Kind regards, Solkrin

mathiasbynens commented 10 months ago

“With today's server save, […] we have fixed the issue that the Despor kill statistics were counted twice […]”

— https://www.tibia.com/news/?subtopic=newsarchive&id=7714

TODO: retroactively correct the data in this repo by halving Despor kill counts up until 2023-10-08. Update: Uneven kill counts would previously show up, e.g. https://github.com/tibiamaps/tibia-kill-stats/blob/ae5abb59c460ec4c2d972b8fd295dc1abbc193ad/data/_global-total/bosses/nimmersatts-breeding-ground-boss-kills.json#L8, presumably when Despor got killed as one of the minibosses during the fight but killing the final boss didn’t succeed. For this reason we cannot accurate correct the data, so we have to keep it as is.

“Despor” got renamed to “Dragon Pack” in today’s client update, so maybe we do need to change the name, however.

muriloricci commented 10 months ago

I checked back with our development team and can tell you that there is indeed an issue with the Despor kill statistic that causes it differ from the kills of the other dragons. At the moment we are working on a fix that will correct this problem. Please understand that I can't tell you when the fix will be released. But I can assure you that this will be done as soon as possible.

Literally received this email one hour before today's server save. :P

muriloricci commented 10 months ago

Rotten Blood echoes are not displayed in kill statistics. Unsure whether this is intentional or not.

This death has been spotted on Gentebra: https://www.tibia.com/community/?subtopic=characters&name=Drakcule+Cheguei+Tarde image

mathiasbynens commented 10 months ago

CipSoft seems to have fixed these two as of 2024-01-31:

  • boar mans should be boar men. fixed
  • crape mans should be crape men. fixed
mathiasbynens commented 5 months ago

Another incorrect pluralization in the kill stats: gingerbread mans should be gingerbread men.

muriloricci commented 4 months ago

The Primal Menace, magma bubble, Wrath of the Emperor and incredibly old witch are now correctly displayed in the kill statistics.

https://www.tibia.com/news/?subtopic=newsarchive&id=7989

Update by @mathiasbynens: This only seems to affect new entries after 2024-07-30. The kill statistics currently contain entries for magma bubble (old) and Magma Bubble (new), for example. I’d suggest waiting until the old entries no longer show up, and then running the scripts to retroactively unify the names.

muriloricci commented 1 month ago

I don't know when this was fixed but Rotworm Queen is now listed properly and not in plural form.