search

osrsbox / osrsbox-db

A complete and up-to-date database of Old School Runescape (OSRS) items, monsters and prayers
https://www.osrsbox.com/projects/osrsbox-db/
GNU General Public License v3.0
227 stars 83 forks source link

Duplicate monsters in monsters api #146

Closed nwalsh1995 closed 5 years ago

nwalsh1995 commented 5 years ago

Hi @osrsbox, its been a while!

Taking a look at the new (to me) monster API and I was noticing some monsters appear to be duplicated. For instance, 8953 & 8954:

MonsterDefinition(id=8953, name='Iorwerth Archer', incomplete=False, members=True, release_date='2004-09-20', combat_level=90, size=1, hitpoints=105, max_hit=11, attack_type=['ranged', 'ranged'], attack_speed=5, aggressive=False, poisonous=False, immune_poison=False, immune_venom=False, weakness=['melee', 'magic'], slayer_monster=True, slayer_level=1, slayer_xp=105, slayer_masters=['vannaka', 'chaeldar', 'nieve', 'duradel'], examine='She looks pretty handy with that bow.', wiki_name='Iorwerth Archer (Female)', wiki_url='https://oldschool.runescape.wiki/w/Iorwerth_Archer#Female', attack_level=10, strength_level=10, defence_level=80, magic_level=1, ranged_level=90, attack_stab=0, attack_slash=0, attack_crush=0, attack_magic=0, attack_ranged=0, defence_stab=50, defence_slash=70, defence_crush=70, defence_magic=60, defence_ranged=50, attack_accuracy=0, melee_strength=0, ranged_strength=0, magic_damage=0, drops=[MonsterDrop(id=526, name='Bones', members=True, quantity='1', noted=False, rarity='1/1', drop_requirements=None), MonsterDrop(id=555, name='Water rune', members=True, quantity='70', noted=False, rarity='8/128', drop_requirements=None), MonsterDrop(id=561, name='Nature rune', members=True, quantity='12', noted=False, rarity='5/128', drop_requirements=None), MonsterDrop(id=563, name='Law rune', members=True, quantity='2', noted=False, rarity='3/128', drop_requirements=None), MonsterDrop(id=554, name='Fire rune', members=True, quantity='37', noted=False, rarity='2/128', drop_requirements=None), MonsterDrop(id=1099, name="Green d'hide chaps", members=True, quantity='1', noted=False, rarity='3/128', drop_requirements=None), MonsterDrop(id=1135, name="Green d'hide body", members=True, quantity='1', noted=False, rarity='4/128', drop_requirements=None), MonsterDrop(id=1243, name='Mithril spear', members=True, quantity='1', noted=False, rarity='2/128', drop_requirements=None), MonsterDrop(id=1197, name='Mithril kiteshield', members=True, quantity='1', noted=False, rarity='1/128', drop_requirements=None), MonsterDrop(id=1161, name='Adamant full helm', members=True, quantity='1', noted=False, rarity='1/128', drop_requirements=None), MonsterDrop(id=1213, name='Rune dagger', members=True, quantity='1', noted=False, rarity='1/128', drop_requirements=None), MonsterDrop(id=617, name='Coins', members=True, quantity='20', noted=False, rarity='5/128', drop_requirements=None), MonsterDrop(id=617, name='Coins', members=True, quantity='44', noted=False, rarity='29/128', drop_requirements=None), MonsterDrop(id=617, name='Coins', members=True, quantity='132', noted=False, rarity='8/128', drop_requirements=None), MonsterDrop(id=617, name='Coins', members=True, quantity='180', noted=False, rarity='10/128', drop_requirements=None), MonsterDrop(id=617, name='Coins', members=True, quantity='440', noted=False, rarity='1/128', drop_requirements=None), MonsterDrop(id=6102, name='Teleport crystal (1)', members=True, quantity='1', noted=False, rarity='1/128', drop_requirements=None), MonsterDrop(id=365, name='Bass', members=True, quantity='1', noted=False, rarity='3/128', drop_requirements=None), MonsterDrop(id=385, name='Shark', members=True, quantity='1', noted=False, rarity='3/128', drop_requirements=None), MonsterDrop(id=449, name='Adamantite ore', members=True, quantity='1', noted=False, rarity='2/128', drop_requirements=None), MonsterDrop(id=13480, name='Ensouled elf head', members=True, quantity='1', noted=False, rarity='1/50', drop_requirements=None), MonsterDrop(id=2722, name='Clue scroll (hard)', members=True, quantity='1', noted=False, rarity='1/128', drop_requirements=None)], rare_drop_table=False)
MonsterDefinition(id=8954, name='Iorwerth Archer', incomplete=False, members=True, release_date='2004-09-20', combat_level=90, size=1, hitpoints=105, max_hit=11, attack_type=['ranged', 'ranged'], attack_speed=5, aggressive=False, poisonous=False, immune_poison=False, immune_venom=False, weakness=['melee', 'magic'], slayer_monster=True, slayer_level=1, slayer_xp=105, slayer_masters=['vannaka', 'chaeldar', 'nieve', 'duradel'], examine='She looks pretty handy with that bow.', wiki_name='Iorwerth Archer (Female)', wiki_url='https://oldschool.runescape.wiki/w/Iorwerth_Archer#Female', attack_level=10, strength_level=10, defence_level=80, magic_level=1, ranged_level=90, attack_stab=0, attack_slash=0, attack_crush=0, attack_magic=0, attack_ranged=0, defence_stab=50, defence_slash=70, defence_crush=70, defence_magic=60, defence_ranged=50, attack_accuracy=0, melee_strength=0, ranged_strength=0, magic_damage=0, drops=[MonsterDrop(id=526, name='Bones', members=True, quantity='1', noted=False, rarity='1/1', drop_requirements=None), MonsterDrop(id=555, name='Water rune', members=True, quantity='70', noted=False, rarity='8/128', drop_requirements=None), MonsterDrop(id=561, name='Nature rune', members=True, quantity='12', noted=False, rarity='5/128', drop_requirements=None), MonsterDrop(id=563, name='Law rune', members=True, quantity='2', noted=False, rarity='3/128', drop_requirements=None), MonsterDrop(id=554, name='Fire rune', members=True, quantity='37', noted=False, rarity='2/128', drop_requirements=None), MonsterDrop(id=1099, name="Green d'hide chaps", members=True, quantity='1', noted=False, rarity='3/128', drop_requirements=None), MonsterDrop(id=1135, name="Green d'hide body", members=True, quantity='1', noted=False, rarity='4/128', drop_requirements=None), MonsterDrop(id=1243, name='Mithril spear', members=True, quantity='1', noted=False, rarity='2/128', drop_requirements=None), MonsterDrop(id=1197, name='Mithril kiteshield', members=True, quantity='1', noted=False, rarity='1/128', drop_requirements=None), MonsterDrop(id=1161, name='Adamant full helm', members=True, quantity='1', noted=False, rarity='1/128', drop_requirements=None), MonsterDrop(id=1213, name='Rune dagger', members=True, quantity='1', noted=False, rarity='1/128', drop_requirements=None), MonsterDrop(id=617, name='Coins', members=True, quantity='20', noted=False, rarity='5/128', drop_requirements=None), MonsterDrop(id=617, name='Coins', members=True, quantity='44', noted=False, rarity='29/128', drop_requirements=None), MonsterDrop(id=617, name='Coins', members=True, quantity='132', noted=False, rarity='8/128', drop_requirements=None), MonsterDrop(id=617, name='Coins', members=True, quantity='180', noted=False, rarity='10/128', drop_requirements=None), MonsterDrop(id=617, name='Coins', members=True, quantity='440', noted=False, rarity='1/128', drop_requirements=None), MonsterDrop(id=6102, name='Teleport crystal (1)', members=True, quantity='1', noted=False, rarity='1/128', drop_requirements=None), MonsterDrop(id=365, name='Bass', members=True, quantity='1', noted=False, rarity='3/128', drop_requirements=None), MonsterDrop(id=385, name='Shark', members=True, quantity='1', noted=False, rarity='3/128', drop_requirements=None), MonsterDrop(id=449, name='Adamantite ore', members=True, quantity='1', noted=False, rarity='2/128', drop_requirements=None), MonsterDrop(id=13480, name='Ensouled elf head', members=True, quantity='1', noted=False, rarity='1/50', drop_requirements=None), MonsterDrop(id=2722, name='Clue scroll (hard)', members=True, quantity='1', noted=False, rarity='1/128', drop_requirements=None)], rare_drop_table=False)

Admittedly I popped both of these reprs into an online diff tool and it didn't find any differences besides their IDs. I'm interested in de-duplicating, am I missing something or do duplicates exist?

nwalsh1995 commented 5 years ago

After looking through these and assuming this isn't a bug from the gathering side, it seems like these could just be recolors/model changes and their difference is not apparent from the stats side.

osrsbox commented 5 years ago

Hey @nwalsh1995 - yes, you are correct that there are duplicates that exist in the monsters dataset. Pretty sure it is not a bug, as the base data is extracted directly from the cache. The situation is similar to the item duplicates issue where multiple items exist and have varying uses in-game (item duplicates for quests, skill menu entries, interface interfaces etc.). I fixed this for items in #94 by adding a duplicate property with a True/False boolean value. The same solution could be implemented for monsters - but would obviously need to be tweaked.

Brief summary: The general idea is achieved using a "first come, first served" principle. The lowest monster (based on monster ID) is considered the "original" when multiple monsters with the same properties are found. The properties that are used for comparison need to be investigated, but could include:

The same solution for items seems to work pretty well. So it seems like a good choice. Also, I am not in favor of removing these duplicate entries (same position as the items database) - as it has been useful to have some of these duplicates available. So if there is a decent solution to filter duplicates out by adding the duplicate property, it seems like the most logical solution.

Edit: Wording in last paragraph.