aerospike / aerospike-client-java

Aerospike Java Client Library
Other
236 stars 212 forks source link

Map or List putItems operation is so slow #97

Closed whting closed 6 years ago

whting commented 6 years ago

When Map or List size is bigger than 500,the operation is so slow. it cost >20ms。

here is my code MapPolicy mapPolicy = new MapPolicy(MapOrder.KEY_ORDERED, MapWriteMode.UPDATE); MapOperation.putItems(mapPolicy, bin, map); In AS, i control map size <1000 and value size <200kb .

xorphox commented 6 years ago

Some questions: Is storage mode data-in-memory? What version of aerospike server are you using? Am i understanding right that you are trying to add 500 elements? And element size < 200 kbytes? Is the bin already there? If it was, is it a KEY_ORDERED map bin?

whting commented 6 years ago

@xorphox 1、storage mode is data-in-memory.I have 2 same node (32C , 128G).here is my config

namespace ns1 {
    replication-factor 2
    memory-size 57G
    default-ttl 90d
    high-water-memory-pct 80
    storage-engine memory
}

2、aerospike server version is 3.15.1.4 , java-client version is 4.0.8

3、I had add 500 elements,and total size <200 kbytes

//first,i add elements at head.Map is order by key.
//and then,i remove index >500
 MapPolicy mapPolicy = new MapPolicy(MapOrder.KEY_ORDERED, MapWriteMode.UPDATE);
client.operate(wp, key, MapOperation.putItems(mapPolicy, bin, map),MapOperation.removeByIndexRange(bin,500, MapReturnType.COUNT));

4、Map bin is a KEY_ORDERED

Can you give me some suggestion?

xorphox commented 6 years ago

putItems(and other CDT ops) have been optimized/improved in the 4.x server versions. If possible try the newest version and see if there is an improvement.

How many items were already in the existing map bin when you added the 500 additional items?

whting commented 6 years ago

@xorphox ok,I will install the latest version map bin had already 500 items,and add one or more items(general one)

xorphox commented 6 years ago

My testing indicates the new version is much faster than the old. Please let us know how that goes. Thank you.

whting commented 6 years ago

I install new version aerospike has an error I download .tar and unzip and make init

error: install -m 644 -o root -g root -C /home/admin/aerospike/aerospike-server/share/udf/lua/external/*.lua /home/admin/aerospike/aerospike-server/share/udf/lua/external/.
xorphox commented 6 years ago

I'm not sure what make init does. Are you trying to compile from the git repo? What exactly is being done?

xorphox commented 6 years ago

Maybe you need to be root to do that.

xorphox commented 6 years ago

QE is fixing this problem. They have told me the following so you can get moving. The work-around is: Comment out line 180 of "share/libexec/aerospike-init" and re-run the "init" command.