aerospike / aerospike-client-java

Aerospike Java Client Library
Other
236 stars 212 forks source link

Aerospike Exception in ScanAll, Error code -11 #180

Closed saurabhkohli-ril closed 3 years ago

saurabhkohli-ril commented 3 years ago

We are trying to scan all the records in a set, and getting an exception as below with the scanall api. AerospikeClient version: 5.0.0

For 1 record, we are getting 5 records because the code is retrying 5 times, this is handled by setting the maxRetries=0, so we get only 1 record.

Tried with both single server and aerospike cluster, getting exception with both. Exception::

com.aerospike.client.query.PartitionTracker.isComplete(PartitionTracker.java:257) at com.aerospike.client.command.ScanExecutor.scanPartitions(ScanExecutor.java:70) at com.aerospike.client.AerospikeClient.scanAll(AerospikeClient.java:1361) at rjil.udmp.jioudmp.utils.UDMPCommonMethods.getAllKeyNames(UDMPCommonMethods.java:1848) at rjil.udmp.jioudmp.Verticles.DBVerticle.plmn.GetAllPlmnCommand.runCommand(GetAllPlmnCommand.java:35) at rjil.udmp.jioudmp.Verticles.DBVerticle.DataBaseVerticle.lambda$start$0(DataBaseVerticle.java:37) at io.vertx.core.eventbus.impl.HandlerRegistration.deliver(HandlerRegistration.java:276) at io.vertx.core.eventbus.impl.HandlerRegistration.handle(HandlerRegistration.java:254) at io.vertx.core.eventbus.impl.EventBusImpl$InboundDeliveryContext.next(EventBusImpl.java:578) at io.vertx.core.eventbus.impl.EventBusImpl.lambda$deliverToHandler$5(EventBusImpl.java:537) at io.vertx.core.impl.ContextImpl.executeTask(ContextImpl.java:320) at io.vertx.core.impl.WorkerContext.lambda$wrapTask$0(WorkerContext.java:34) at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76) at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)

Scan code::

ScanPolicy policy = new ScanPolicy(); policy.maxRetries=0; JsonArray array = new JsonArray(); try{ aerospikeClient.scanAll(policy, UdmpConstants.AEROSPIKE_NAMESPACE_NAME, tableName, new ScanCallback() { @Override public void scanCallback(Key key, Record record) throws AerospikeException { array.add(record.getString(UdmpDBConstants.NAME_BIN)); } }, UdmpDBConstants.NAME_BIN); } catch (AerospikeException e) { udmpBootStrapper.writeLog(Level.INFO, "UdmpCommonMethods.getAllKeyNames", e.getMessage(), ""); }

BrianNichols commented 3 years ago

Scan retries are attempted on network connection errors or timeouts. The MAX_RETRIES_EXCEEDED exception is shown when maxRetries is exhausted. The underlying error can be determined by adding a log statement here:

https://github.com/aerospike/aerospike-client-java/blob/c1f005ebd71738df08c2e99c7a806f7ea069c7a7/client/src/com/aerospike/client/command/ScanPartitionCommand.java#L64

Also, the scan callback code needs to be thread-safe for scans that concurrently retrieve data from different nodes.

saurabhkohli-ril commented 3 years ago

Added the logs, no additional logging was rcvd.

I suspect the issue to be in this line(PartitionTracker.isComplete):: if (partsReceived >= partsRequested || (maxRecords > 0 && recordCount >= maxRecords)) {

partsRequested: 4096 partsReceived: 0 recordCount: 1 maxRecords: 0

When we set the maxRecords to 1 in the scanPolicy, it passes because of the second check(maxRecords > 0 && recordCount >= maxRecords), but exception again comes when there is no record in the DB.

BrianNichols commented 3 years ago

Which server version are you using?

This scan functionality requires server version 4.9+.

saurabhkohli-ril commented 3 years ago

Thanks, We upgraded the server and it worked.