ropensci / elastic

R client for the Elasticsearch HTTP API
https://docs.ropensci.org/elastic
Other
245 stars 58 forks source link

incorrect scroll example #228

Closed Neleawan closed 5 years ago

Neleawan commented 5 years ago

Hello, In the documentation and the vignette, the scroll example is incorrect. At least for my case with elasticsearch 2.3

The example is the following;

res <- Search(index = 'shakespeare', q="a*", time_scroll="5m")
out <- list()
hits <- 1
while(hits != 0){
    res <- scroll(res$`_scroll_id`)
    hits <- length(res$hits$hits)
    if(hits > 0)
      out <- c(out, res$hits$hits)
}

But by doing so, since the default size of Search is 10, you miss 10 results from the first Search. Actually, the first Search return a scroll id for the next results. And I think that if you add the time_scroll in the first Search, you should as well in the scroll.

So I think that the example should be something like this, in order to retrieve all the results:

res <- Search(index = 'shakespeare', q="a*", time_scroll="5m")
out <- list()
hits <- length(res$hits$hits)
if(hits > 0){
    out <- c(out, res$hits$hits)
    while(hits != 0){
        res <- scroll(res$`_scroll_id`,  time_scroll="5m")
        hits <- length(res$hits$hits)
        if(hits > 0){
            out <- c(out, res$hits$hits)
        }
    }
}
sckott commented 5 years ago

thanks for the issue @Neleawan - I'll have a look and get back to you

sckott commented 5 years ago

what version of elastic R package are you using?

Neleawan commented 5 years ago

I'm using 0.8.0

sckott commented 5 years ago

i think this commit fixes the egs, let me know if ot