Baseline Test new ArcGIS Map Server sync performance

[ReedHunter-Wsdot]

As part of our efforts to ...

• make sync slowdowns as rare as possible
• Migrate to the latest Esri server software
• rid our prod servers of the webcontextUrl setting without blocking map syncs
• estimate what kind of servers we'll need these next few years to support HATS

... this task is to create a baseline load test result of three VM servers using the latest version of ArcGIS Server. This task will be completed when we find out how many simultaneous syncs against these servers degrade response time to at least twice what it is without heavy load.

Steps:

1. setup three VM servers in the latest version of ArcGIS Server, a file server for the config store, and a web adaptor. Completed

2. setup a copy of the Hats beta map service on these servers Completed

3. use the .net Hats app generate an offline replica using this service, confirming everything is configured ok, Completed

4. run about ten low load tests - one sync at a time. Use this to make a baseline of an average sync time on the test servers. We can use Hats Beta for this step, or a modified copy of the offline feature map generator. Completed Results with just one test server: If the geodatabase is ...

   • ... already synced with the server, then the average sync time was ~68 seconds
   • ... missing about 1 month's worth of data from field crews, average sync is ~230 seconds.

5. modify a copy of that .net offline features map generator app to instead work as a load test client. Completed

6. Load test the vm servers to confirm their sync speeds when under heavy load, simulating a typical rush hour morning. Find out how many simultaneous map sync requests will cause map sync times to slow down to taking at least twice the baseline, low load speed. Completed - count of simultaneous clients is 22 to double sync time from a single client.

The QA virtual server syncs will be a mixed bag compared to our current production environment syncs, maybe it impossible to predict or explain why they may be slower or faster.

Factors making them potentially faster are ...

• newer software. We don't know if newer releases are more optimized.
• servers are not shared with other map services for different business areas
• the test app will not make edits in between each sync, while in production there are constantly field crews making edits. The test syncs will therefore be "skinny" in content compared to a typical production map sync. I can access records on Qa map syncs for better comparison.

Factors making them potentially slower are ...

• they're virtual machines, and VMs have a reputation of taking a 5% to 10%performance hit compared to physical machine performance
• the physical machine(s) they're running on may be in use by other business areas. We may be able to make arrangements with server admins and/or varying the time of day of the tests to make sure this isn't an issue.
• special server hardware is currently allocated for our production environment. Those servers are likely inherently faster than the VM host hardware.

Next Steps

• once we have the QA baseline low and heavy load test parameters, we can run it during off hours such as Friday around noon on our production servers to compare Qa vs Prod performance.
• we can also run the low load test on single VM vs hardware servers to estimate the cost of virtualization on map syncs

[ReedHunter-Wsdot]

Closing this since server testing is done and we decided the VMs will work.