Automatic batching of larger CSV data imports

lxnow / philippines-erpnext

0 stars 0 forks source link

Automatic batching of larger CSV data imports #5

Open lxnow opened 10 years ago

lxnow commented 10 years ago

Currently data imports of CSV files are limited to smaller files (200-500 rows depending what form is being updated). This requires user to cut up the file several times, making sure not to miss any data rows and avoid duplication. Takes too much time and prone to human error.

To improve this experience, have a middleware tool sitting at the server hidden from the user that will take on larger CSVs and automatically chop it up and feed in batches to erpnext. This will all be done server-side.

rmehta commented 10 years ago

This is a bit tricky

[ ] create Batch Upload doctype - one attachment and submit
[ ] create a scheduler event to break and import
[ ] send html report as email and update Batch Upload

Estimate: 1 day