johnlpage
commented
2 years ago Have around 300 million to 500 million documents in a collection. Run POC Driver for a while, shouldn't take that long to load depending on size and hardware.
Around 10,000 to 20,000 servers connecting to Mongo at once. That's a pretty bad and unusual anti-pattern if you mean 10K actual servers not connections but if you do then run it on 10,000 to 20,000 servers and make sure you use the --threadIdStart flag correctly.
Each of those servers passes in about 100,000 to 200,000 unique keys.
Don't really know what you mean by that
Mongo projects from the document and returns values for each of those keys to each of those servers.
They want a target of half a second response time per server. Load is very bursty where all servers are requesting resources at once.
I suggest contacting MongoBD Professional services - this sounds like a poor fit in terms of calls and schema usage - it's certainly one they would want expert tuning for. 20,000 requests each for 100,000 separate document retrieved by key is likely to be very I/O Bound just seeking and fetching those documents.
The forums at mongodb.com/developer are also a much better place to ask questions like this - this is for Bugs in POCDriver only.
does anyone know how I would do the following with the POC driver?
I’m thinking to generate the 500 million documents with a json generator first…. I think I can simulate the 20k servers with -t option… however I don’t see any options to do random find() 200k times within one call.