timshannon / badgerhold

BadgerHold is an embeddable NoSQL store for querying Go types built on Badger
MIT License
515 stars 52 forks source link

Question: how to select what parts of the record to pull #18

Closed davedbase closed 4 years ago

davedbase commented 4 years ago

I may have gotten this wrong, but I'm noticing that if I have a record that is a complex type badgerhold tend to retain the full data structure into memory. Sometimes I'm just want to perform a count or select a single value on the record. Consider this struct:

type Document struct {
    ID         uuid.UUID `badgerhold:"index"`
    Reference  string    `badgerhold:"index"`
    Title      string
    Excerpt    []byte
    Type       string `badgerhold:"index"`
    Status     string
    Properties []Property
    Date       time.Time
    Modified   time.Time
}

If I run this command:

store.Find(&results, badgerhold.Where("Type").Eq("speakers").Index("Type"))

I get 400 or so records but the query takes 93ms to run. When I print the values the WHOLE body is being pulled. At first this scared me but then I realised it pulled far more than I required. Is there I way I can only extract certain properties?

I'd imagine if I restricted the amount of data to extract that performance will increase dramatically. Most of the heft in my record is in the Properties element. It's about 5x the size of the actual struct.

Apologies if this is documented. I'm trying to tip toe into Badger and Badgerhold. :-)

timshannon commented 4 years ago

Badgerhold is just a small layer on top of Badger DB, which is a Key Value store. The Value is the entire struct, so the whole value of the struct is brought out of the file into memory and used for checking against the criteria of the query.

There is no way currently to pick and choose fields.