aerospike / aerospike-client-java

Aerospike Java Client Library
Other
236 stars 212 forks source link

Expose low level API #43

Closed alexandrnikitin closed 9 years ago

alexandrnikitin commented 9 years ago

I want to avoid data convertions that happen when client converts policy, namespaces, set names, keys to byte array. I.e. in Command and in Key Instead I want to see something like

public byte[] get(byte[] policy, byte[] key) throws AerospikeException;
// or
public byte[] get(byte[] policy, byte[] namespace, byte[] set, byte[] keyHash) throws AerospikeException;

Whant do you think?

BrianNichols commented 9 years ago

Providing very low-level access restricts our ability to make future modifications because these routines would then be fixed to avoid breaking user code. The serialization to byte[] is less of a factor than TCP/IP latency, so it's unlikely that this will be supported in the near future.

alexandrnikitin commented 9 years ago

Sorry, I won't agree with you. I work on high load projects where a server handles ~200K RPS and we're going to >400K. On those volumes it counts, every allocation counts. That's why I want to have low level API. I started with some benchmarks using JMH in benchmark-get branch. I removed all socket interactions and record parsing logic. The changes in PR. It gives me ~400K RPS per core which isn't much imho.

alexandrnikitin commented 9 years ago

Quick update: I added multithreaded version of benchmark but it doesn't scale linearly. I got ~2.5 increase for 4 threads. And if I simply remove key creation and comment out conversion logic I get ~5x increase up to 2M RPS.

That's my intention, to have a client with as less overhead as possible. And if Aerospike position itself as a high performance database then I want to see that in clients too not only in database server.