I think I stumbled upon a technical limitation while comparing large (but similar to a very large extend) CSV files.
❯ daff 80921.20240125.120915.280075/stop_times.txt 80921.20240125.180818.104400/stop_times.txt
/Users/thbar/.asdf/installs/nodejs/16.17.1/lib/node_modules/daff/bin/daff.js:11375
throw(e);
^
Error: Cannot create a string longer than 0x1fffffe8 characters
at Object.slice (node:buffer:599:37)
at Buffer.toString (node:buffer:819:14)
at Object.readFileSync (node:fs:506:41)
at Object.tio.getContent (/Users/thbar/.asdf/installs/nodejs/16.17.1/lib/node_modules/daff/bin/daff.js:11195:15)
at $hx_exports.coopy.Coopy.loadTable (/Users/thbar/.asdf/installs/nodejs/16.17.1/lib/node_modules/daff/bin/daff.js:2359:21)
at $hx_exports.coopy.Coopy.run (/Users/thbar/.asdf/installs/nodejs/16.17.1/lib/node_modules/daff/bin/daff.js:2933:17)
at $hx_exports.coopy.Coopy.coopyhx (/Users/thbar/.asdf/installs/nodejs/16.17.1/lib/node_modules/daff/bin/daff.js:3014:15)
at run_daff_base (/Users/thbar/.asdf/installs/nodejs/16.17.1/lib/node_modules/daff/bin/daff.js:11326:18)
at Object.daff.run_daff_main (/Users/thbar/.asdf/installs/nodejs/16.17.1/lib/node_modules/daff/bin/daff.js:11343:13)
at Object.<anonymous> (/Users/thbar/.asdf/installs/nodejs/16.17.1/lib/node_modules/daff/bin/daff.js:11361:11) {
code: 'ERR_STRING_TOO_LONG'
}
The files are closer from 750MB, and the limit seems to be around 0x1fffffe8 = 512MB.
I think I stumbled upon a technical limitation while comparing large (but similar to a very large extend) CSV files.
The files are closer from 750MB, and the limit seems to be around 0x1fffffe8 = 512MB.
I wonder if this is can be fixed easily or not !