zemirco / json2csv

Convert json to csv with column titles
http://zemirco.github.io/json2csv
MIT License
2.72k stars 362 forks source link

Performance optimizations #491

Closed juanjoDiaz closed 4 years ago

juanjoDiaz commented 4 years ago

This undoes the performance optimizations that were introduced in #360 and #378.

It seems that v8 has improved significantly: support is a lot better and native implementation of the methods are consistently much faster in all browsers, Node, and Deno.

It can be easily tested using:

function flattenReducer(acc, arr) {
  try {
    // This is faster but susceptible to `RangeError: Maximum call stack size exceeded`
    acc.push(...arr);
    return acc;
  } catch (err) {
    // Fallback to a slower but safer option
    return acc.concat(arr);
  }
}

let start = performance.now();
Array(100).fill(Array(1000).fill(1)).reduce(flattenReducer);
let end = performance.now();
console.log(`Time: ${end - start} ms.`);

start = performance.now();
Array(100).fill(Array(1000).fill(1)).flat();
end = performance.now();
console.log(`Time: ${end - start} ms.`);
function fastJoin(arr, separator) {
  let isFirst = true;
  return arr.reduce((acc, elem) => {
    if (elem === null || elem === undefined) {
      elem = '';
    }

    if (isFirst) {
      isFirst = false;
      return `${elem}`;
    }

    return `${acc}${separator}${elem}`;
  }, '');
}

let size = 1000
let start = performance.now();
fastJoin(Array(size).fill('test'), '-');
let end = performance.now();
console.log(`Time: ${end - start} ms.`);

start = performance.now();
Array(size).fill('test').join('-');
end = performance.now();
console.log(`Time: ${end - start} ms.`);

for Node, you need to import performance doing const { performance } = require('perf_hooks');

coveralls commented 4 years ago

Coverage Status

Coverage remained the same at 100.0% when pulling 7f357767311bc0c08f00704a5aaf843f479f7b3c on juanjoDiaz:performance_optimizations into 4c74c1e661e8bd0e4a5ff0efa2ba22a34ea9da35 on zemirco:master.

knownasilya commented 4 years ago

I wonder if we could add a perf test to see if things change with updates or just with node updates

juanjoDiaz commented 4 years ago

There is #365 about perf tests. I've just never found something like coveralls but for performance. To compare different engines and to avoid regressions. I'm open to ideas.

We should merge this anyway unless there is any blocker. I have some other PRs coming after this.