search

PhilanthropyDataCommons / data-scripts

Command-line scripts to add data to a PDC service instance.
GNU Affero General Public License v3.0
1 stars 0 forks source link

Race between row and EOF events for forms and proposals #25

Open bickelj opened 1 year ago

bickelj commented 1 year ago

In generateApplicationFormJson.ts and a similar future script for proposals, when using the csv-reader library, there appears to be a race between .on('data',... row event and on('end'... end of file event such that if row processing takes some time, the end of input file event can occur before all rows are processed and the overall processing ends prematurely.

bickelj commented 1 year ago

Using https://csv.js.org/parse/api/async_iterator/ appears to resolve this, see #40.

bickelj commented 1 year ago

Resolved with the merge of #40 (sync API, not async iterator).

bickelj commented 1 year ago

Oh wait, that resolution only applied to the post proposals script, not the rest.