ostafen / clover

A lightweight document-oriented NoSQL database written in pure Golang.
MIT License
666 stars 55 forks source link

Cursor for running through big datasets. #63

Closed trading-peter closed 2 years ago

trading-peter commented 2 years ago

Hi,

I'm trying out clover for an easy and portable way of storing stock market price data. And it works good enough for my usecase in general. The only thing I was wondering is if there is support for cursors? Right now I'm doing a FindAll() which takes pretty long and obv. causes the whole chunk of data to be stored in memory.

It would be awesome if something like this was possible

cond := clover.Field("openTime").GtEq(tw.Start).And(clover.Field("openTime").LtEq(tw.End))
cursor, err := s.db.Query("candles").Where(cond).Sort(clover.SortOption{"openTime", 1}).Cursor()

for cursor.Next() {
  cursor.Unmarshal(&myStruct)

  // do something with myStruct
}
ostafen commented 2 years ago

Hi, @trading-peter. You can use the ForEach() method for this purpose, which calls a supplied function for each document

trading-peter commented 2 years ago

Ooooh so nice! How did I miss that hahaha. Thanks you!