search

aerospike / aerospike-client-go

Aerospike Client Go
Apache License 2.0
433 stars 199 forks source link

BatchGetComplex API returns PARAMETER_ERROR when there is migration going on and allowPartialResult is setting as True #322

Closed juvenzhu closed 3 years ago

juvenzhu commented 3 years ago

We are using BatchGetComplex API to perform batchGet. Recently, we found when there is migration going on on the server side, the BatchGetComplex API will return error (PARAMETER_ERROR). The error will go away once the migration has been finished.

The client version we are using is 2.12.0 and 3.0.3

juvenzhu commented 3 years ago

@khaf Can you help us take a look! Thanks

khaf commented 3 years ago

Of course. I need a bit of time to be able to reproduce the issue. I'll have to ping our server team to see if this is in fact unexpected behavior as well.

clusc commented 3 years ago

Thanks @khaf. This issue is rather important to us, it's been impacting our service SLA everytime there's a server event causing migrations. Do you prefer us to create a corresponding ticket on the support time to expedite the process?

khaf commented 3 years ago

Creating a support ticket would help, since I think we'll need help from other teams to diagnose the issue. At this point while I have some idea of why this can happen, I'm not completely positive if that's the case and what can be done to mitigate the problem.

khaf commented 3 years ago

Can you also report the values you use for the policy?

juvenzhu commented 3 years ago

Thanks

I have opened a support ticket. This is the policy we are using:

The values for the batchPolicy are: -ConcurrentNodes: 0 -AllowInline: true -AllowPartialResults: true -basePolicy:   - TotalTimeout: 500ms  - SocketTimeout:250ms   - MaxRetries: 3   - SleepBetweenRetries: 100ms   - SleepMultiplier: 2   - SendKey: true

kportertx commented 3 years ago

Many errors returned to the client also generate a warning message in the server logs. Could you check if such a warning appears and provide the message here?

juvenzhu commented 3 years ago

@khaf @kportertx this turned out to be a server issue and the new release fixed this.

Thanks a lot for helping.

khaf commented 3 years ago

No problem, we were working on the issue behind the scene and helped the server team in debugging the issue. Feel free to open new issues any time.