search

forcedotcom / phoenix

BSD 3-Clause "New" or "Revised" License
558 stars 227 forks source link

Pig Suppport - BatchSize option Issue #617

Open abraxas73 opened 10 years ago

abraxas73 commented 10 years ago

Hello, I raise this issue some time ago, I don't think it is solved.

Below is the issue content that I raised several month ago.

Please go over again.

After several experiment, I found out that records which has smaller number than "batchSize" value is ignored and they cannot be stored.

For example) I have 8759 records to be stored, I set the batchSize option value to 1000. Then, below script just store 8000 records to HBase.

STORE ACCOUNT_LOGINOUT_HB INTO 'hbase://ACCOUNT_LOGINOUT' using com.salesforce.phoenix.pig.PhoenixHBaseStorage('how-bmzookeeper01','-batchSize 1000');

Is this behavior as desgned?

So, I have to set batchSize option value to "1", to store all of my records.

jtaylor-sfdc commented 10 years ago

What version of Phoenix are you using, as this issue was fixed some time ago.

abraxas73 commented 10 years ago

I checked this again with 2.0.2.

jtaylor-sfdc commented 10 years ago

Please try with 2.2.0 and if there's still a problem, then we'll fix right away.

Thanks, James

On Dec 8, 2013, at 3:30 PM, abraxas73 notifications@github.com wrote:

I checked this again with 2.0.2.

— Reply to this email directly or view it on GitHubhttps://github.com/forcedotcom/phoenix/issues/617#issuecomment-30096457 .