pk_chunking is not working

[hrishy]

Hi

I tried the example in the documentation

job = bulk.create_query_job("Contact", contentType='CSV', pk_chunking=True)
batch = bulk.query(job, "select Id,LastName from Contact")
bulk.close_job(job)
while not bulk.is_batch_done(batch):
    sleep(10)

for result in bulk.get_all_results_for_query_batch(batch):
    reader = unicodecsv.DictReader(result, encoding='utf-8')
    for row in reader:
        print(row) # dictionary rows

BulkBatchFailed Traceback (most recent call last)

in 8 batch = bulk.query(job, "select Id,LastName from Contact") 9 bulk.close_job(job) ---> 10 while not bulk.is_batch_done(batch): 11 sleep(10) 12 c:\python37\lib\site-packages\salesforce_bulk\salesforce_bulk.py in is_batch_done(self, batch_id, job_id) 426 if batch_state in bulk_states.ERROR_STATES: 427 status = self.batch_status(batch_id, job_id) --> 428 raise BulkBatchFailed(job_id, batch_id, status.get('stateMessage'), batch_state) 429 return batch_state == bulk_states.COMPLETED 430 BulkBatchFailed: Batch 123456789 of job None failed: None Without pk_chunking it works though. Any ideas what am i doing wrong here any help would be appreciated

[kunal4422]

Hi @hrishy did u find solution to this?

[hrishy]

Pk_chunking actually refers to chunk_size and the default value is 2048 I used vs code ide to figure this out

On Mon 23 Dec, 2019, 11:48 AM kunal4422, notifications@github.com wrote:

Hi @hrishy https://github.com/hrishy did u find solution to this?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heroku/salesforce-bulk/issues/74?email_source=notifications&email_token=AJCOEWPWM2BQO44YHTPOJJDQ2BJ3FA5CNFSM4IO3G36KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHQKJXA#issuecomment-568370396, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJCOEWOA2IAUEY6WJOIH6GLQ2BJ3FANCNFSM4IO3G36A .

[kunal4422]

@hrishy didnt got you. I am having same code as you and also same error. Instead of pk_chunking=True i tried to give pk_chunking=100000; as given in example. What exactly should i do to rectify this error? Somewhere we have to use get_batch_items() for iterating between batches. How did u solve it?

[hrishy]

Hi

My apologies I don't have acesses to my code now but I do remember vaguely that I used visual studio code to explore the method defination and that is when I came to know that pk_chunk is actually chunk_size in the method and default value is 2048 if you don't specify it

On Mon 23 Dec, 2019, 3:48 PM kunal4422, notifications@github.com wrote:

@hrishy https://github.com/hrishy didnt got you. I am having same code as you and also same error. Instead of pk_chunking=True i tried to give pk_chunking=100000; as given in example. What exactly should i do to rectify this error? Somewhere we have to use get_batch_items() for iterating between batches. How did u solve it?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/heroku/salesforce-bulk/issues/74?email_source=notifications&email_token=AJCOEWNAKRQ2CVV7WVWDMOTQ2CF7FA5CNFSM4IO3G36KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEHQZHHA#issuecomment-568431516, or unsubscribe https://github.com/notifications/unsubscribe-auth/AJCOEWMEFWLYWLY53WNCPDLQ2CF7FANCNFSM4IO3G36A .

[roshin8]

I added an example to the README which shows how to use PK Chunking. Let me know if that helps you. https://github.com/heroku/salesforce-bulk/pull/77/files

[fungss]

I added an example to the README which shows how to use PK Chunking. Let me know if that helps you. https://github.com/heroku/salesforce-bulk/pull/77/files

This should be merged into README. It works really well. Thanks a lot.