timshannon / badgerhold

BadgerHold is an embeddable NoSQL store for querying Go types built on Badger
MIT License
514 stars 52 forks source link

Unnecessary usage of iterator #77

Closed b-tarczynski closed 2 years ago

b-tarczynski commented 2 years ago

When using index query with operator eq or in it would be better to use simple badger.Get to get value of that index rather than using iterator which iterate over every index and decode value of KeyList.

    type TestStruct struct {
        Value string
        Index int `badgerhold:"index"`
    }

    for i := 0; i < 10_000; i++ {
        testStruct := TestStruct{
            Value: strconv.Itoa(i),
            Index: i,
        }
        err := s.storage.database.Badger.Insert(i, &testStruct)
        s.NoError(err)
    }

    results := make([]TestStruct, 0, 1)
    err := s.storage.database.Badger.Find(
        &results,
        bh.Where("Index").Eq(5000).Index("Index"),
    )
    s.NoError(err)
    s.Len(results, 1)
    s.Equal(results[0].Index, 5000)

In above test badgerhold.Store iterates over all records and calls Decode function 10k times. It should be just badger.Get call for badgerhold.KeyList and one get for each value in that struct.

msieczko commented 2 years ago

Hey @timshannon, I would go further and label this issue as a bug. Iterating over all index keys (PrefetchValues enabled), instead of doing a simple index query in this case, seems to kill performance benefits of the index.