jeroen / mongolite

Fast and Simple MongoDB Client for R
https://jeroen.github.io/mongolite/
284 stars 64 forks source link

iter obj skip limit keyword unexpected results #248

Open featherduino opened 1 year ago

featherduino commented 1 year ago

Error: Failed to send "getMore" command with database Failed to read 7028797 bytes: socket error or timeout

jeroen commented 1 year ago

Did you try the iterator: https://jeroen.github.io/mongolite/query-data.html#iterating

featherduino commented 1 year ago

Yup this is after using the iterator

Stored the iter obj in a variable

Then appended in a list using iter$one command

featherduino commented 1 year ago
z<-list()
it<-temp_stats$iterate(query='{"action": "CREATE_POST","_id": {"$gt": {"$oid":"63ebd3280000000000000000"}}}',limit=5000,skip = 1000)
while(!is.null(it$one())){
 rlist::list.append(z,it$one())
}

FYI

featherduino commented 1 year ago

okay i got this to work but am getting half the records which is surprising count of records for query in DB ~ 50 K I get 23K

featherduino commented 1 year ago

There seems to be a bug because when i used it<-temp_stats$iterate(query='{"action": "CREATE_POST","_id": {"$gt": {"$oid":"63ebd3280000000000000000"}}}',skip = 26885)

I get 40k rows including the first 1000 ones when i did not add skip