Then convert the base64 data to binary and parse using the arrow JavaScript library. Then pass the arrow Table as the values property of the dataset. Arrow Tables support the array-of-objects format that Vega uses, so this mostly just works. But I've run into a few issues:
Arrow Int columns are converted to JavaScript BigInt values, which don't play well with Vega
Arrow Tables don't support mutation, and in one example I tried there is a mutation error raised (the example is at vl-convert-rs/tests/vl-specs/circle_binned_b64.vl.json).
test test_png_no_theme::test::case_02 ... ERROR TypeError: 'set' on proxy: trap returned falsish for property 'bin_maxbins_10_IMDB_Rating'
at https://cdn.skypack.dev/-/vega-transforms@v4.10.2-ZwuxbjLubhdUwIKHKtlL/dist=es2020,mode=imports,min/optimized/vega-transforms.js:1:11242
at Array.forEach (<anonymous>)
Given these issues, I'm not sure it makes sense to move forward with this right now. But wanted to document my findings.
This PR is an experiment towards supporting base64-encoded arrow IPC data. The idea is to support data URLs of the form:
Then convert the base64 data to binary and parse using the arrow JavaScript library. Then pass the arrow Table as the
valuesproperty of the dataset. Arrow Tables support the array-of-objects format that Vega uses, so this mostly just works. But I've run into a few issues:vl-convert-rs/tests/vl-specs/circle_binned_b64.vl.json).Given these issues, I'm not sure it makes sense to move forward with this right now. But wanted to document my findings.