search

pyexcel / pyexcel-io

One interface to read and write the data in various excel formats, import the data into and export the data from databases
http://io.pyexcel.org
Other
58 stars 20 forks source link

DjangoBookWriter does not support batch_size #24

Closed readflybird closed 7 years ago

readflybird commented 7 years ago

In class DjangoBookWriter, the open_content() method, the parameters keywords are ignored and thus there is no way we can pass in the batch_size. Besides, the create_sheet() method does not take care of batch_size, neither. Therefore there is no way we can do a batch insert through DjangoModelWriter.

chfw commented 7 years ago

It is true that 'batch_size' was not passed on and 'batch_size' controls the number of records being inserted at once. If not mentioned, default batch_size 999 is used. However, it won't block batch insertion.

Please check for database error such as integrity error during bulk import. Once there is any exception, it will default to 'snail' alike insertion - one record at a time, skipping offending ones.

readflybird commented 7 years ago

Thanks for the quick fix. Is there any plan that to merge this change into a release branch?

chfw commented 7 years ago

by default, I would release it with v0.3.0, which should be released by the end of this year. Can you wait for it? Otherwise, I can do a quick release as v0.2.6.

chfw commented 7 years ago

released in v0.2.6 via pypi

readflybird commented 7 years ago

Thanks a lot for the response! I have tried out the new version and it works great!