Add sandbox queue management when using buildSandboxPerVisit

[kratiahuja]

In order to increase the efficiency of the system when using buildSandboxPerVisit flag, we currently lazily create the sandbox for the next request. This is a single sandbox that is leveraged. In order to help with performance, we will be maintaining a queue of these sandboxes that can be leveraged by the requests. When the sandbox is empty, it will eagerly create the sandbox instead of caching the pending request and letting it run when the sandbox is available. This is primarily so that we don't end up starving the system when number of requests is far greater than queue size and the system cannot recover from storing the sandbox for next requests. Finishing the request (even if in degraded fashion) brings in better resilience.

In addition, when requests cannot use the sandbox from the queue, we now emit analytics information about the request in the result object. This helps applications using this flag to track if their QPS is greater than Queue size and bump the queue size.

The queue size is provided when the app is created and pre-emptively the queue is filled up so initial requests don't starve.

Updated README as well.

cc: @rwjblue @melmerp

Next:

• [ ] Add visit timing information in analytics of the result object

[kratiahuja]

Also ran some perf numbers against the tests and they are in-line to the previous PR here when sandbox is being leveraged from a previous request.