Spreads / Spreads.LMDB

Low-level zero-overhead and the fastest LMDB .NET wrapper with some additional native methods useful for Spreads
http://docs.dataspreads.io/spreads/libs/lmdb/api/README.html
Mozilla Public License 2.0
80 stars 9 forks source link

IntegerDuplicates works with ulong but not with long #23

Closed aliostad closed 5 years ago

aliostad commented 5 years ago

When I use long for values of duplicates, it is not able to find them when I TryFindDup. It is able to find for int and ulong.

Can you please explain?

I am planning to use long (since is a more natural type) in my design. I can go to ulong but just need to understand whether the support for ulong is only perhaps because of supporting only at int level.

buybackoff commented 5 years ago

Could you please post a repro? Do you have negative longs?

aliostad commented 5 years ago

I am so sorry, I cannot reproduce it anymore. It could be because I was truncating a database that previously had int? that is weird.

buybackoff commented 5 years ago

I have just added a test and it works:

[Test]
public void CouldWriteLongDups()
{
    var path = TestUtils.GetPath();
    var env = LMDBEnvironment.Create(path, DbEnvironmentFlags.WriteMap | DbEnvironmentFlags.NoSync);

    env.MapSize = 100 * 1024 * 1024;
    env.Open();

    var db = env.OpenDatabase("dupfixed_db",
        new DatabaseConfig(DbFlags.Create | DbFlags.IntegerDuplicates));
    ulong count = 100;

    using (Benchmark.Run("Long Wrt+TFD", (long)count))
    {
        for (ulong i = 1; i < count; i++)
        {
            var key = 0;
            var value = i;
            try
            {
                db.Put(0, i, TransactionPutOptions.AppendDuplicateData);

                using (var txn = env.BeginReadOnlyTransaction())
                {
                    if (!db.TryFindDup(txn, Lookup.EQ, ref key, ref value))
                    {
                        Assert.Fail("!db.TryGet(txn, ref key, out value)");
                    }

                    if (value != i)
                    {
                        Assert.Fail($"value {value} != i {i}");
                    }
                }
            }
            catch (Exception e)
            {
                Console.WriteLine(e.ToString());
            }
        }
    }

    Benchmark.Dump();
    db.Dispose();
    env.Dispose();
}
buybackoff commented 5 years ago

It could be because I was truncating a database that previously had int? that is weird.

I believe you should drop such database. Fixed size is probably saved after first write and used after that.

MDB_INTEGERKEY Keys are binary integers in native byte order, either unsigned int or size_t, and will be sorted as such. The keys must all be of the same size. http://www.lmdb.tech/doc/group__mdb.html#gac08cad5b096925642ca359a6d6f0562a