search

aerospike / aerospike-client-java

Aerospike Java Client Library
Other
236 stars 212 forks source link

Not working attribute "recordsPerSecond" in Policy #185

Closed timewizhan closed 3 years ago

timewizhan commented 3 years ago

Hi. When using scan with AerospikeClient, I found a weird problem. I figured out recordsPerSecond is used to manage/control the number of records through this document "https://www.aerospike.com/apidocs/java/com/aerospike/client/policy/ScanPolicy.html"

However, I realized that the attribute has not been affected to the scan despite I adjusted the value for it.

val policy = ScanPolicy()
policy.recordsPerSecond = 300

And I cannot find codes related to this attribute in AerospikeClient. I think it is not implemented yet. Otherwise, let me know how to use it plz... :)

Thanks

BrianNichols commented 3 years ago

Which server version are you using?

timewizhan commented 3 years ago

Sorry. I'm late I am using version 5 :)

BrianNichols commented 3 years ago

recordsPerSecond was introduced in server version 4.7, so it should be supported on your server. The client does send the field on scans when recordsPerSecond > 0. Can you provide a code snippet of your scan call?

timewizhan commented 3 years ago 
private val policy = ScanPolicy()

    init {
        policy.concurrentNodes = true
        policy.priority = Priority.LOW
        policy.includeBinData = true
        policy.recordsPerSecond = 300
    }
...

aerospikeClient.use { aerospikeClient ->
            aerospikeClient.scanAll(policy, namespace, table, this)
        }

This code is similar to the snippet provided in the sample, in which I just added "recordsPerSecond" attribute.

BrianNichols commented 3 years ago

I tried the following on a set of 500,001 records and it does limit the scan to ~10,000 records per second.

ScanPolicy policy = new ScanPolicy();
policy.recordsPerSecond = 10000;
client.scanAll(policy, params.namespace, params.set, this);

Output:

2021-04-15 12:37:42 PDT INFO Add node BB96F695D290C00 172.16.70.197 3000
2021-04-15 12:37:42 PDT INFO ScanParallel Begin
2021-04-15 12:37:42 PDT INFO Scan parallel: namespace=test set=demoset
2021-04-15 12:37:43 PDT INFO Records 10000
2021-04-15 12:37:44 PDT INFO Records 20000
2021-04-15 12:37:45 PDT INFO Records 30000
2021-04-15 12:37:46 PDT INFO Records 40000
2021-04-15 12:37:47 PDT INFO Records 50000
2021-04-15 12:37:48 PDT INFO Records 60000
2021-04-15 12:37:49 PDT INFO Records 70000
2021-04-15 12:37:50 PDT INFO Records 80000
2021-04-15 12:37:51 PDT INFO Records 90000
2021-04-15 12:37:52 PDT INFO Records 100000
2021-04-15 12:37:53 PDT INFO Records 110000
2021-04-15 12:37:54 PDT INFO Records 120000
2021-04-15 12:37:55 PDT INFO Records 130000
2021-04-15 12:37:56 PDT INFO Records 140000
2021-04-15 12:37:57 PDT INFO Records 150000
2021-04-15 12:37:58 PDT INFO Records 160000
2021-04-15 12:37:59 PDT INFO Records 170000
2021-04-15 12:38:00 PDT INFO Records 180000
2021-04-15 12:38:01 PDT INFO Records 190000
2021-04-15 12:38:02 PDT INFO Records 200000
2021-04-15 12:38:03 PDT INFO Records 210000
2021-04-15 12:38:04 PDT INFO Records 220000
2021-04-15 12:38:05 PDT INFO Records 230000
2021-04-15 12:38:06 PDT INFO Records 240000
2021-04-15 12:38:07 PDT INFO Records 250000
2021-04-15 12:38:08 PDT INFO Records 260000
2021-04-15 12:38:09 PDT INFO Records 270000
2021-04-15 12:38:10 PDT INFO Records 280000
2021-04-15 12:38:11 PDT INFO Records 290000
2021-04-15 12:38:12 PDT INFO Records 300000
2021-04-15 12:38:13 PDT INFO Records 310000
2021-04-15 12:38:14 PDT INFO Records 320000
2021-04-15 12:38:15 PDT INFO Records 330000
2021-04-15 12:38:16 PDT INFO Records 340000
2021-04-15 12:38:17 PDT INFO Records 350000
2021-04-15 12:38:18 PDT INFO Records 360000
2021-04-15 12:38:19 PDT INFO Records 370000
2021-04-15 12:38:20 PDT INFO Records 380000
2021-04-15 12:38:21 PDT INFO Records 390000
2021-04-15 12:38:22 PDT INFO Records 400000
2021-04-15 12:38:23 PDT INFO Records 410000
2021-04-15 12:38:24 PDT INFO Records 420000
2021-04-15 12:38:25 PDT INFO Records 430000
2021-04-15 12:38:26 PDT INFO Records 440000
2021-04-15 12:38:27 PDT INFO Records 450000
2021-04-15 12:38:28 PDT INFO Records 460000
2021-04-15 12:38:29 PDT INFO Records 470000
2021-04-15 12:38:30 PDT INFO Records 480000
2021-04-15 12:38:31 PDT INFO Records 490000
2021-04-15 12:38:32 PDT INFO Records 500000
2021-04-15 12:38:32 PDT INFO Total records returned: 500001
2021-04-15 12:38:32 PDT INFO Elapsed time: 50.025 seconds
2021-04-15 12:38:32 PDT INFO Records/second: 9995.0
2021-04-15 12:38:32 PDT INFO ScanParallel End