Adds a new Parameter Server, PartitionedParameterServer. The server supports atomic, in-order processing of push and pull operations, using multiple threads.
Renames the existing "Single Node" Server as ConcurrentParameterServer.
I've done a simple performance comparison between PartitionedPS and ConcurrentPS by running the unit tests (1million reads + 1million writes) on my local laptop. The PartitionedPS has worse performance. Note that the ConcurrentPS can lose updates.
ConcurrentPS: Avg 1.5s
PartitionedPS: Avg 2s
Also, the unit test does not include hash computation. A version of the test (not committed) that uses MurmurHash had an Avg runtime of 3s. However, this test also requires the key to be copied into a byte array, which we avoid in the actual implementation by using the original serialized value from the message.
Closes #176
I've done a simple performance comparison between PartitionedPS and ConcurrentPS by running the unit tests (1million reads + 1million writes) on my local laptop. The PartitionedPS has worse performance. Note that the ConcurrentPS can lose updates.
Also, the unit test does not include hash computation. A version of the test (not committed) that uses MurmurHash had an Avg runtime of 3s. However, this test also requires the key to be copied into a byte array, which we avoid in the actual implementation by using the original serialized value from the message.