Open drewbitt opened 2 months ago
Yeah... this part can be confusing. The tl;dr is that writeParquet
frees the table itself. We should probably clarify this in the function's docstring
Functions exported from rust through wasm-bindgen can either take inputs by reference or by value, and the latter consumes the input object. Here, writeParquet
takes the input table by value, and so consumes its data.
You can always check the __wbg_ptr
property of a wasm object to check whether the data has been freed or not. If the pointer is 0, it's a null pointer and the data has already been freed.
> let wasm = require('parquet-wasm/node')
> let properties = new wasm.WriterPropertiesBuilder().build()
undefined
> properties.__wbg_ptr
2621480
> properties.free()
undefined
> properties.__wbg_ptr
0
Thank you! That was helpful
I think adding that to the docstring and not erroring when this happens - stopping all execution - would be nice to have. A console.warn would be more suitable.
not erroring when this happens - stopping all execution - would be nice to have
That's not something I can control. That's part of the auto-generated bindings by rust's wasm-bindgen.
Let's keep this open as a reminder to improve the documentation here
Reproduction
Output
``` tsx json-parquet-2.ts Table { schema: Schema { fields: [ [Field], [Field] ], metadata: Map(0) {}, dictionaries: Map(1) { 0 => [Utf8] }, metadataVersion: 4 }, batches: [ RecordBatch { schema: [Schema], data: [Data] } ], _offsets: Uint32Array(2) [ 0, 2 ] } Table { __wbg_ptr: 2369000 } /Users/drewbitt/Repos/Pantomath/benchmarking/node_modules/parquet-wasm/node/parquet_wasm.js:3359 throw new Error(getStringFromWasm0(arg0, arg1)); ^ Error: null pointer passed to rust at module.exports.__wbindgen_throw (/Users/drewbitt/Repos/x/benchmarking/node_modules/parquet-wasm/node/parquet_wasm.js:3359:11) at wasm://wasm/014c002a:wasm-function[6573]:0x405d03 at wasm://wasm/014c002a:wasm-function[6574]:0x405d10 at wasm://wasm/014c002a:wasm-function[3297]:0x3a06de at wasm://wasm/014c002a:wasm-function[4074]:0x3c8bd1 at Table.free (/Users/drewbitt/Repos/x/benchmarking/node_modules/parquet-wasm/node/parquet_wasm.js:2095:14) atI'm not very well aligned in this space, so let me know if this is expected for some reason. Thanks!