skale-me / node-parquet

NodeJS module to access apache parquet format files
Apache License 2.0
57 stars 11 forks source link

CLI: numeric filename converted to number, raises javascript TypeError #49

Open pappasam opened 6 years ago

pappasam commented 6 years ago

First: thank you for building this super-useful tool. Definitely comes in handy when needing to run quick diffs against two parquet files.

Issue

Numeric filenames cause javascript errors. Since my parquet files are output by hive, they have numeric names.

Details

If you run the following code:

parquet head 00000

You'll get the following error message:

cat 0
/home/youruser/.nodenv/versions/6.11.2/lib/node_modules/node-parquet/bin/parquet.js:54
  const reader = new parquet.ParquetReader(file);
                 ^

TypeError: wrong argument
    at TypeError (native)
    at cat (/home/sroeca/.nodenv/versions/6.11.2/lib/node_modules/node-parquet/bin/parquet.js:54:18)
    at Object.<anonymous> (/home/sroeca/.nodenv/versions/6.11.2/lib/node_modules/node-parquet/bin/parquet.js:43:5)
    at Module._compile (module.js:570:32)
    at Object.Module._extensions..js (module.js:579:10)
    at Module.load (module.js:487:32)
    at tryModuleLoad (module.js:446:12)
    at Function.Module._load (module.js:438:3)
    at Module.runMain (module.js:604:10)
    at run (bootstrap_node.js:389:7)

Workaround

At present, the simple workaround is to rename the files to a non-numeric value. This is mildly cumbersome.

pappasam commented 6 years ago

bin/parquet.js will most-likely need some major refactoring. Shouldn't be too difficult to implement, but I'm not sure what you're planning for this module in the future.

mvertes commented 6 years ago

Thanks for your report. I fixed it in my branch, correction will be included in the next release.

Regarding plans for this module, I intend to improve it to the point where I can convert parquet files from/to JSON and CSV format. I would like also to have a working pure JS version, which will be embeddable in browser. I'm lacking time at this moment, but hope to improve it anyway

mvertes commented 6 years ago

was unvoluntary closed. Sorry. The state is: corrected, waiting for validation