delta-io / delta-kernel-rs

A native Delta implementation for integration with any query engine
Apache License 2.0
153 stars 42 forks source link

Handle nested columns #86

Open roeap opened 11 months ago

roeap commented 11 months ago

We need to handle nested columns in various locations, data skipping being an obvious one.

We should come up with a robust and simple design that is capable of handling column mapping and other special scenarios as well.

Known problems to solve:

nicklan commented 4 months ago

@roeap can you clarify what's not properly handled today? afaict here we will traverse into nested columns.

So is there something more general you're talking about here?