Invalid non-string/buffer chunk parseAsync

[garethquirke]

1. Latest version 2.Chrome and on Node 12.4.0

  const fields = columns;
    const opts = { fields };

    const asyncParser = new AsyncParser(opts);

    let csv = '';
    asyncParser.processor
    .on('data', chunk => console.log(chunk)) //with or without this line same result, it logs a Uint8Array
    .on('data', chunk => (csv += chunk.toString()))
    .on('end', () => console.log(csv))
      .on('error', err => console.error(err));

      asyncParser.transform
      .on('header', header => console.log(header))
      .on('line', line => console.log(line))
      .on('error', err => console.log(err));

    asyncParser.input.push(data.data); // This data might come from an HTTP request, etc.
    asyncParser.input.push(null);

    return csv;

4. Sample data

[{category: "Ice cream", flavour: "vanilla"}, {category: "Ice cream", flavour: "chocolate"}]

5. Error: Invalid non-string/buffer chunk

[juanjoDiaz]

Everything looks correct on your sample and the error also looks correct.

If the content of data.data is not a string or a Buffer, you can't pass it to the AsyncParser. I assume that you are using a library that parses the response of your request into JSON for you which is the problem.

Possible solutions: 1.- Use object mode:

const asyncParser = new AsyncParser(opts, { objectMode: true });
...
data.data.forEach(item => asyncParser.input.push(item));
asyncParser.input.push(null);

It's easy but you loose the advantage of streamming the data as it comes and the constant memory pressure.

2.- Use the convenience method parse async:

parseAsync(data.data, opts);

It does the exact same thing as option 1 under the hood.

3.- Stringify the JSON

...
asyncParser.input.push(JSON.stringify(data.data));
asyncParser.input.push(null);

4.- Actually stream the data. This depends on which HTTP library you are using. If you are using this in node.js, Response is already a stream and can be passed as input asyncParser.fromInput(res). If you are on the browser, you need to get the chunks of data as they arrive and push them to the parser.

[garethquirke]

Thanks for the reply, the first option solved my problem. Wondering if there is a method to modify a row value during the parsing process. For example if I had an ID column and wanted to wrap the ID in a URL, thanks

[juanjoDiaz]

Unfortunately, not yet (at least in a very clean way). It's something that I've been thinking of for a while so I've created #406 to add it for v5.

It should be possible for you anyway since you are using objectMode. However, there is a bug in the current library that prevents it from working. I'll try to push the fix later today and I'll give you an example of how to do it.

[garethquirke]

I managed to find another way around this task, thanks :)