Initial csv batch limiting functionality

CityOfLosAngeles / hcid-lookup-tool

A lookup tool for data from LAHD. Repository has been archived 2022-10-13.

Apache License 2.0

0 stars 3 forks source link

Closed rhouse00 closed 7 years ago

rhouse00 commented 7 years ago

Parsing of csv into batches to allow for subsequent processing

Allows for data flow control, in case of stream interruption

Todo:

condition needed for when batching is done, but under the row minimum. something akin to parser.end() == true
deletion of temp csv file rows, as rows are inserted into batch array
If stream interruption, how to start back up and at where we left off