Mobs dropping too many items

[Deadwing888]

A mob (even one with more than ten possible drops) should never drop more than ten items.

It is also generally accepted that Treasure Hunter (especially TH4) is overpowered on Darkstar. I don't know the specifics of how it's currently coded but I would say it's overpowered by at least a factor of two. I have a feeling I'm about to receive a "tag you're it" to do some treasure hunter parsing and I'm willing to take that on if nobody has the goods on treasure hunter mechanics.

It's also possible that drops seem more common because treasure hunter is getting baked in twice. For instance wiki reports the drop rate for Herald's Gaiters at 25%, but Tiamat would almost always be killed with TH4 on... I think in this fashion it's possible that treasure hunter is already baked into the wiki drop rates for some items and then gets applied again when there is a thief present.

Also Dynamis armor drops are more common than they should be, but it's hard to say if that's OP TH talking or drop rates. Relic currency in Dynamis Jeuno is similarly too common.

[teschnei]

IIRC, every level of treasure hunter adds an extra chance for every drop, which is definitely not how it works on retail. TH13 absolutely does not give you 13 chances for an item. I think it's been thought to just give a flat 1% bonus chance for th3+. You can just look at the ffxidb drop rates for data

[TeoTwawki]

@teschnei I want to add a column to the drop table: drop slot. most of the loot pool overflows come from not just abnormally high drop rates but the fact both items are dropping independently when they should not be. And with permission, I'll change TH to being +chance instead of re-rolls on our imaginary dice.

[TeoTwawki]

@teschnei unless you want either-or type drops done a diff way. Then I am all ears. I had an alternative worked out in case you didn't want a new column for it, but I forgot wtf it was now.

[teschnei]

nah thats probably fine

On Wed, Sep 30, 2015 at 10:35 PM, TeoTwawki notifications@github.com wrote:

@teschnei https://github.com/teschnei unless you want either-or type drops done a diff way. Then I am all ears. I had an alternative worked out in case you didn't want a new column for it, but I forgot wtf it was now.

— Reply to this email directly or view it on GitHub https://github.com/DarkstarProject/darkstar/issues/2158#issuecomment-144618037 .

[towbes]

Seems like adding the drop slot column is a solution on its own, why change the re-rolls aspect as well? The research done here (before all the TH5+ changes) http://khoisan.livejournal.com/15302.html indicates that the re-roll aspect is indeed correct. Perhaps research at current live levels would shed some light on TH5+ being more chance% increase.

On Thu, Oct 1, 2015 at 11:06 AM, Tyler Schneider notifications@github.com wrote:

nah thats probably fine

On Wed, Sep 30, 2015 at 10:35 PM, TeoTwawki notifications@github.com wrote:

@teschnei https://github.com/teschnei unless you want either-or type drops done a diff way. Then I am all ears. I had an alternative worked out in case you didn't want a new column for it, but I forgot wtf it was now.

— Reply to this email directly or view it on GitHub < https://github.com/DarkstarProject/darkstar/issues/2158#issuecomment-144618037

.

— Reply to this email directly or view it on GitHub https://github.com/DarkstarProject/darkstar/issues/2158#issuecomment-144618145 .

[TeoTwawki]

@towbes not what that data looks like to me. Note the smaller diff's between tiers as it increases, even as overall drop % goes up, but Looks more like diminishing returns for higher values of TH. Which explains why going from TH3 to 4 feels like a placebo for so many players, Yet reaching TH2 is pretty noticeable over no TH at all on stuff like the example bees. The author himself say this.

TH0: You get about 40 items per 100 kills TH1: You get about 60 items per 100 kills. ~ 50% increase over TH0. TH2: You get about 80 items per 100 kills. ~ 100% increase over TH0 and ~ 33% over TH1. TH3: You get about 88 items per 100 kills. ~ 120% increase over TH0 and ~ 10% over TH2. TH4: You get about 95 items per 100 kills. ~ 137% increase over TH0 and ~ 8% over TH3.

Which may suggest that neither flat "1 th lv = 1%" boost nor "re-rolls" is the retail way it happens. (assuming the data was reliably tested and a large enough sample size, which it looks be but I am not done reading it)

Whatever way I get told it should be by @teschnei is what I'll do (if anything) to it.

[towbes]

Just to clarify how it's currently working from ai_mob_dummy.cpp

---

Mob dies, game pulls it's drop list and puts it in an array, then determines the size of that array. For loop Begin Set tries to 0 for item slot Set the max number of tries equal 1 plus the max level of TH present when mob died Set the bonus rate (what is this???)

Start the while loop for each item while tries < maxTries grab a random number between 0-1000 if random number is less than the drop rate * (drop rate multiplier) + bonus add item to the treasure pool break out of the while loop else add 1 to tries

Go back to beginning of For loop for next item

So the drop rate multiplier is set by the server admin in map config, what is the TH bonus rate that is calculated and added to the drop rate?

[Deadwing888]

I had always assumed the "treasure hunter grants more rolls" theory was just invented and tossed around by people who didn't have a good grasp on how the server might actually be handling random numbers.

Indeed looking at this data I don't really see it holding water. If there are 4 possible drops and TH0 -> TH1 gave you "another roll" then you would expect to see the drop rate double the way it apparently currently does on Darkstar. If it does something other than double then "extra rolls" starts to break down because if you only get these "extra rolls" part of the time, then suddenly maybe this isn't the best way to be thinking about it. (or maybe from a coding perspective it is?)

Keeping in mind that this is out of 400 possible drops over the 100 kills the data shows having a base of 10% of possible drops TH0 leads to: 15% TH1 > 20% TH2 > 22% TH3 > 23.75% TH4. I buy that. The sample size is a bit low, but I buy it.

Juxtaposed against Darkstar's (if I understand this correctly) 10% > 20% > 30% > 40% > 50% by the time you get to TH4 it's off by about a factor of two which is... what I had originally eyeballed. That, Towbes, is why we should look into changing the re-rolls aspect.

[teschnei]

well... 5 drops at 10% is 1 - (0.9_0.9_0.9_0.9_0.9) (41%), but close enough

On Thu, Oct 1, 2015 at 1:03 AM, Deadwing888 notifications@github.com wrote:

I had always assumed the "treasure hunter grants more rolls" theory was just invented and tossed around by people who didn't have a good grasp on how the server might actually be handling random numbers.

Indeed looking at this data I don't really see it holding water. If there are 4 possible drops and TH0 -> TH1 gave you "another roll" then you would expect to see the drop rate double the way it apparently currently does on Darkstar. If it does something other than double then "extra rolls" starts to break down because if you only get these "extra rolls" half the time, then suddenly maybe this isn't the best way to be thinking about it. (or maybe from a coding perspective it is?)

Keeping in mind that this is out of 400 possible drops over the 100 kills the data shows having a base of 10% of possible drops TH0 leads to: 15% TH1

20% TH2 > 22% TH3 > 23.75% TH4. I buy that. The sample size is a bit low, but I buy it.

Juxtaposed against Darkstar's (if I understand this correctly) 10% > 20% > 30% > 40% > 50% by the time you get to TH4 it's off by about a factor of two which is... what I had originally eyeballed. That, Towbes, is why we should look into changing the re-rolls aspect.

— Reply to this email directly or view it on GitHub https://github.com/DarkstarProject/darkstar/issues/2158#issuecomment-144638989 .

[Deadwing888]

Ah yes, because the rolls don't help you once the item has decided to drop.

[towbes]

I agree it needs to be looked at, but I just don't think flat rate increases are the right answer. Just a peek through some 5k+ count mobs in ffxidb shows an almost 100% increase from TH0->TH1, but TH1->TH2 is substantially lower in some cases

http://ffxidb.com/zones/105/stalking-sapling

http://ffxidb.com/zones/105/sabertooth-tiger

Then there is data that is completely twisted after TH1

http://ffxidb.com/zones/147/platinum-quadav

http://ffxidb.com/zones/147/sapphire-quadav

Even those high sample sizes don't seem to have the best data, but there does seem to be a trend of diminishing returns post TH1

[TeoTwawki]

If its diminishing returns like I thought looking at that 1st link, what we might need in order to simulate the retail data, is have the TH tiers not be just +1 mod TH each, or do some math against the TH value in core, before applying it to the drop chance directly.

..Some items in the ffxiddb websites data really throw me off (I guess I never really looked at it comparing 2 items % before, instead of just finding a single items %). I see tiger hides actually go up in steps, defying the diminishing return theory. And I see tiger fangs just JUMP to near 50 then stay approximately the same with small steps up...Hmm. Th1, double, Th2+ small bump up? Meh. Nothing seems to fit perfectly, leading me right back to "wtf IS retail even DOING? gah!"

[towbes]

This guys comment at the bottom of the TH research livejournal post is interesting, is along the lines of what you are talking about I think deadwing, where it is extra roll chances, but not necessarily a full drop chance:

A couple thoughtsfrom: anonymous date: Dec. 15th, 2009 05:41 am (UTC) Link http://khoisan.livejournal.com/15302.html?thread=72134#t72134 Nice work so far. After reading this and personal experiences, I had a few thoughts I figured I'd throw out there.

I'm starting to get the impression drops are tiered off and mobs work similar to armory crates. When the mob dies, the system does a series of randoms for certain value ranges on items with caps on the number of rolls for each category and number of items that can drop from each category with a total cap of 10 items (as SE has claimed) before going to a final set for crystals.

From the looks of your research, I'm going to guess that Treasure Hunter adds to the possible number of rolls in each category by a % (+50% TH1, +100% TH2, then +20% for each TH+1 piece equipped). This could explain why more common drops are enhanced significantly, while people tend to not see as dramatic improvement on items with very low drop rates.

For example, let's make up a table for Tinnin and rolls needed. Group 1 (100% Drop Rate Items, need to roll 0-999) 1 Roll Each - Max of 1 item drops Tinnin Scale (TH4 makes it 2.4 -> 2 rolls, but it caps at 1 drop)

Group 2 (Hachiryu - Need to Roll 950-999 - 2 Rolls - Cap of 1) TH4 = 4.8 -> 4 rolls. Stops after a successful roll

Group 3 (Weapons - 0-125 Shusui, 126-874 No Drop, 875-999 Alkalurops - 2 Rolls - Cap of 1?) TH4 = 4 rolls

Groups 4 (Hats - 0-250 Oracles, 251-749 No Drop, 750-999 Enkidu, 4 rolls, cap of 2) TH4 = 9.6 -> 9 Rolls, stops after 2 successful rolls

Group 5 (Enkidu's Harness - Need to roll 750-999 - 2 Rolls - Cap of 2) TH4 = 4 rolls

Group 6 (Tinnin's Fan - Need to roll 850-999 - 2 Rolls - Cap of 1) TH4 = 4 Rolls

Drop total counted. Crystal rolls occur to fill in the gaps where things didn't drop, and stops if the crystals earned puts the total drop count at 10.

That's my thoughts at least. It would explain why TH raises tendencies to drop but doesn't always seem to improve items that have a poor drop rate to begin with.

On Thu, Oct 1, 2015 at 8:26 PM, TeoTwawki notifications@github.com wrote:

If its diminishing returns like I thought looking at that 1st link, what we might need in order to simulate the retail data, is have the TH tiers not be just +1 mod TH each, or do some math against the TH value in core, before applying it to the drop chance directly.

..Some items in the ffxiddb websites data really throw me off (I guess I never really looked at it comparing 2 items % before, instead of just finding a single items %). I see tiger hides actually go up in steps, defying the diminishing return theory. And I see tiger fangs just JUMP to near 50 then stay approximately the same with small steps up...Hmm. Th1, double, Th2+ small bump up? Meh. Nothing seems to fit perfectly, leading me right back to "wtf IS retail even DOING? gah!"

— Reply to this email directly or view it on GitHub https://github.com/DarkstarProject/darkstar/issues/2158#issuecomment-144732718 .

[m241dan]

In regards to TH being baked in twice, that is correct and it is why TH seems overpowered on DSP.

In terms of rerolls, unless it has been changed, it's capped at two rerolls currently. Forgive me if I'm repeating what people already know but it seems that someone was indicating DSP code rerolled for each TH. I just wanted to make it clear that you get a reroll for TH1 and TH2 and then 1% increased droprate per TH after that, according to how it is coded now.

Sorry if this was useless information, just wanted it to be clear because on my initial read through it was not.

[TeoTwawki]

@towbes I wouldn't be shocked if it actually does work sort of like armory crates on retail.

@m241dan thats how I thought it was the comments here on github which lead me to believe it was just re-rolling. But now I've looked at the file..this is whats there.

    for (uint8 i = 0; i < DropList->size(); ++i)
    {
        //THLvl is the number of 'extra chances' at an item. If the item is obtained, then break out.
        uint8 tries = 0;
        uint8 maxTries = 1 + (m_PMob->m_THLvl > 2 ? 2 : m_PMob->m_THLvl);
        uint8 bonus = (m_PMob->m_THLvl > 2 ? (m_PMob->m_THLvl - 2)*10 : 0);
        while (tries < maxTries)
        {
            if (dsprand::GetRandomNumber(1000) < DropList->at(i).DropRate * map_config.drop_rate_multiplier + bonus)
            {
                PChar->PTreasurePool->AddItem(DropList->at(i).ItemID, m_PMob);
                break;
            }
            tries++;
        }
    }

As you can see boys and girls, it really is capped on those re-rolls and adding a flat 1% thereafter.

So..I'm leaving it the heck alone unless told to specifically do something about it. Just gonna take care of those either/or drops for now.

[xipies]

In April of this year, I experienced both "no drop" (#1354) and "both drop" from Quu Domi the Gallant, which I think is supposed to be an either/or drop.

[TeoTwawki]

Probably won't have time to test my work and pull request it till friday, but I got started on handling either/or drops via a column for drop slot.

[Deadwing888]

It looks to me like it would best match the live-journal testing if TH1 was a 50% chance to roll again, TH2 was 100% chance to roll again. 1% for each TH after that.

[cszark]

I have tested this code using additional debugging and confirmed that there are too many processes for drops based on what what has been discussed here. I have confirmed the following # of processes on a single drop.

TH0: x1 Proc (i.e. 75war w/ no gear) TH1: x2 Process (i.e. 40thf w/ no gear) TH2: x3 Process (i.e. 75thf w/ no gear) TH3: x3 Process (i.e. 75thf w/ Thief's Knife) + 1%

As an old school career ffxi thief, I tend to believe TH0: x1 Proc TH1: ~50% chance at x2 Proc TH2: ~x2 Proc TH3: adds 1% for anything above TH2

Which specific model is this project trying to aim for? I think multiple models can be added with ease. Getting 10+ 100's clearing Dynamis Sandy doesn't seem like old school retail.

Here is my code for testing. This is not a fix and just provides visibility into processing. I don't natively code in C++, just SQL.

    //Treasure Hunter Processing Debugging (0=off, 1=simple, 2=detailed)
    uint8 debugTHProcLvl = 2;

        DropList_t* DropList = itemutils::GetDropList(m_DropID);
    //debugTHProc Level 1
    if (debugTHProcLvl > 0)
    {
        ShowDebug(CL_CYAN"DropID: %u dropping with TH Level: %u\n" CL_RESET, m_DropID, m_THLvl);
    }
        if (DropList != nullptr && !getMobMod(MOBMOD_NO_DROPS) && DropList->size())
        {
            for (uint8 i = 0; i < DropList->size(); ++i)
            {
                //THLvl is the number of 'extra chances' at an item. If the item is obtained, then break out.
                uint8 tries = 0;
                uint8 maxTries = 1 + (m_THLvl > 2 ? 2 : m_THLvl);
                uint8 bonus = (m_THLvl > 2 ? (m_THLvl - 2) * 10 : 0);
        uint8 skiproll = 0;

        //debugTHProc Level 2
        if (debugTHProcLvl > 1)
        {
            ShowDebug(CL_CYAN"maxTries: %u bonus: %u %\n" CL_RESET, maxTries, bonus);
        }

                while (tries < maxTries)
                {
        uint16 random = dsprand::GetRandomNumber(1000);                        

        //debugTHProc Level 2
        if (debugTHProcLvl > 1)
        {
            ShowDebug(CL_CYAN"SkipRoll: %u ItemID: %u DropRate: %u Random: %u\n" CL_RESET, skiproll, DropList->at(i).ItemID, DropList->at(i).DropRate, random);
        }

        if (skiproll == 0 && DropList->at(i).DropRate > 0 && random < DropList->at(i).DropRate * map_config.drop_rate_multiplier + bonus)
                    {
            PChar->PTreasurePool->AddItem(DropList->at(i).ItemID, this);
            if (debugTHProcLvl < 1)
            {
            break;
            }
            skiproll = 1;
                    }
                    tries++;
                }
            }
        }

[TeoTwawki]

Since this getting a new comment sent me a notice reminding me I never finished it: I broke my branch where I was working on adding "drop slots"/groupings to the mob drops table. If anyone else wants to take a stab at adding and using the new column before I come back to it (kind of demotivated for now), good luck to you.

Just don't do what the bcnm loot system does, it doesn't work correctly.

[cszark]

Thanks. I will work on it. Still not familiar with the whole commit process. I got a working model done last night. If anyone is interested in the source before I publish it, please PM me. I am a github noob to say the least.

[TeoTwawki]

github doesn't have a PM feature.

[TeoTwawki]

forgot this issue was here: the core has the loot groups now and cap to prevent overflow drops, but groupings still need set in the drop tables to make it retail accurate.