Apache Arrow support

[1chandu]

This issue tracks remaining work for columnar table support in deck.gl

deck.gl v8.0

loaders.gl v2.0

• Tracker: Columnar Table Support in loaders.gl v2

GPUTable (@ibgreen/@1chandu)

• [ ] GPUTable documentation (@ibgreen)
• [ ] Arrow column->GPUTable attribute mappings (Arrow metadata semantics) documentation (@ibgreen)
• [ ] GPUTable initial implementation (@ibgreen)
• [ ] GPUTable full implementation (@1chandu)

GPUTable transformations (@1chandu)

• [ ] GPU Table Transformer: Transform (combine etc) raw Apache Arrow columns in GPU tables into Layer-compatible Attribute tables (positions = vec3, colors = vec4s, float64 handling, matrices, ...)

  • Inputs:
    • Schema defining Apache Arrow table.
    • One Buffer object for each Apache table column

  • Outputs:

    • Set of Buffers to match deck.gl Layer attributes.
    • [x] Make GPUTransform work for a single attribute (position column generated by longitude and latitude columns), write unit tests #3796
    • [ ] Integrate GPUTable/GPUTransform with deck.gl and make single example work
    • [ ] Send GPUTable as data prop , and have AttributeManager/Layer extract attributes and set.
    • [ ] Use loaders.gl to generate column data, and use it to build GPUTable
      • [ ] Verify loaders.gl arrow loader with simple arrow data
      • [ ] Use csv loader to generate arrow data
    • [ ] Use 3D heatmap example to generate arrow data from csv file, build GPU table and pass it as data prop.
    • [ ] Move GPUTable and new column generation into deck.gl ??

[1chandu]

Over to @Pessimistress , close it if this is covered by other issues.