Wsm2110 / Faster.Map

Faster.Map is a high-performance, thread-safe key-value store designed to outperform the standard Dictionary and ConcurrentDictionary
MIT License
75 stars 7 forks source link

Sometimes DenseMapSIMD Resize loses entries #29

Closed NightlyRevenger closed 1 year ago

NightlyRevenger commented 1 year ago

Not sure why this is happening.

Minimum working example:

[TestMethod]
public void AnItemHasBeenLost()
{
    List<long> keysList = new List<long>()
    {
        3053810, 6107620, 9161430, 12215240, 39699530, 36645720, 33591910, 30538100,
        27484290, 109937160, 106883350, 103829540, 100775730, 97721920, 82452870, 79399060, 76345250, 73291440, 70237630,
        198497650, 195443840, 192390030, 189336220, 186282410, 171013360, 167959550, 164905740, 161851930, 158798120, 128260020,
        125206210, 122152400, 119098590, 116044780, 94668110, 91614300, 88560490, 296219570, 293165760, 290111950, 287058140,
        284004330, 268735280, 265681470, 262627660, 259573850, 256520040, 225981940, 222928130, 219874320, 216820510, 213766700,
        207659080, 204605270, 201551460, 180174790, 177120980, 174067170, 137421450, 134367640, 131313830, 433641020, 430587210,
        427533400, 424479590, 421425780, 406156730, 403102920, 400049110, 396995300, 393941490, 363403390, 360349580, 357295770,
        354241960, 351188150, 345080530, 342026720, 338972910, 335919100, 332865290, 329811480, 326757670, 323703860, 317596240,
        314542430, 311488620, 308434810, 305381000, 302327190, 299273380, 274842900, 271789090, 253466230, 247358610, 244304800,
        241250990, 238197180, 235143370, 232089560, 229035750, 210712890, 149636690, 146582880, 143529070, 140475260, 601600570,
        598546760, 595492950, 592439140, 589385330, 574116280, 571062470, 568008660, 564954850, 561901040, 531362940, 528309130,
        525255320, 522201510, 519147700, 513040080, 509986270, 506932460, 503878650, 500824840, 497771030, 494717220, 491663410,
        485555790, 482501980, 479448170, 476394360, 473340550, 470286740, 467232930, 464179120, 461125310, 458071500, 455017690,
        451963880, 442802450, 439748640, 436694830, 415318160, 412264350, 409210540, 390887680, 387833870, 384780060, 381726250,
        378672440, 375618630, 372564820, 369511010, 366457200, 348134340, 277896710, 250412420, 781775360, 778721550, 775667740,
        772613930, 769560120, 754291070, 751237260, 748183450, 745129640, 742075830, 711537730, 708483920, 705430110, 702376300,
        699322490, 693214870, 690161060, 687107250, 684053440, 680999630, 677945820, 674892010, 671838200, 665730580, 662676770,
        659622960, 656569150, 653515340, 650461530, 647407720, 644353910, 641300100, 638246290, 635192480, 632138670, 622977240,
        619923430, 616869620, 613815810, 610762000, 607708190, 604654380, 586331520, 583277710, 580223900, 577170090, 558847230,
        555793420, 552739610, 549685800, 546631990, 543578180, 540524370, 537470560, 534416750, 516093890, 488609600, 448910070,
        445856260, 418371970, 320650050, 280950520, 183228600, 992488250, 989434440, 986380630, 983326820, 980273010, 965003960,
        961950150, 958896340, 955842530
    };

    long missingKey = 439748640L;

    Assert.IsTrue(keysList.Contains(missingKey), "Sanity check failed");

    var keysSet = new DenseMapSIMD<long, long>(190);
    keysSet.Emplace(0L, 0L);

    foreach (var key in keysList)
    {
        if (!keysSet.Contains(key))
            keysSet.Emplace(key, key);

        if (key == missingKey)
            Assert.IsTrue(keysSet.Contains(missingKey), "Sanity check failed");
    }

    Assert.IsTrue(keysSet.Contains(missingKey), "Entry has been lost");
}
Wsm2110 commented 1 year ago

Not really sure what your re trying to do here mate. seeing as the missingKey is also in your keyset. Anyhow there is no need to use contains() while emplacing. Enplace will return true or false

NightlyRevenger commented 1 year ago

With current Resize() implementation last Assert fails

Wsm2110 commented 1 year ago

FIxed it. When we use a get() or contains() we always use a rehash when the probing exceeds the current length of the hashmap. While resizing and emplacing this doesn't happen. Added the rehashing while resizing did the trick