nwfsc-fram / OceanTS

Oceanographic data processing in Typescript using NodeJS and Apache Arrow
0 stars 3 forks source link

Finalize parsing of filenames for xmlcon, vessel, year alignment #14

Closed ghost closed 5 years ago

ghost commented 5 years ago

All of the parsing is completed. However I'm still not able to iterate through a listing of files, and then with each file iterate through each line and process the content using async / await. The readline module apparently now supports async / await per:

https://nodejs.org/api/readline.html#readline_rl_symbol_asynciterator

using a sample such as:

async function processLineByLine() {
  const rl = readline.createInterface({
    // ...
  });

  for await (const line of rl) {
    // Each line in the readline input will be successively available here as
    // `line`.
  }
}

but when I try that, I get an error on the rl portion of the for await line:

Type 'Interface' is not an array type or a string type.

ghost commented 5 years ago

still need to separate out the csv files by year and potentially by vessel as well. For the vessel component, I might either put them all into a vessel folder, or potentially merge all of the vessel results for a given year into a single file.

ghost commented 5 years ago

The readline code is now working using the syntax provided above. Also, by using the for ... of syntax the bulkProcess.ts code is now operating in proper order.

Reference: https://lavrton.com/javascript-loops-how-to-handle-async-await-6252dd3c795/

ghost commented 5 years ago

splitting output apart by year and vessel is now in place.