ballerina-platform / ballerina-spec

Ballerina Language and Platform Specifications
Other
167 stars 54 forks source link

Improved support for CSV-like data #1076

Open jclark opened 2 years ago

jclark commented 2 years ago

We should aim make processing of CSV-like data as smooth as processing JSON data.

For example, given a record type R, it should be possible to construct a value of type table<R> or R[] from a CSV-file with a header line, and the HTTP dispatcher should be able to do this for a payload with a text/csv MIME type.

CSV-like means we want to support not just text files in CSV format, but things like Google Sheets where the data model is CSV-like. JSON-support in Ballerina is both support for the JSON as an on the wire format, and support for the json subtype of anydata. We need both levels for CSV.

CSV-like means:

jclark commented 2 years ago

There are quite a number of little things that will combine to make this all nice (some in langlib and some in the standard library). A major part of this is about making it easier to convert between lists and mappings in a type-aware way.

A couple of easy things (not enough but a start):

There's also the data binding aspect. In langlib, I think we should have something like:

jclark commented 2 years ago

If you want to convert from a record r to a tuple in a typed way, you can do it with an expression

let var {x, y z} = r in [x, y, z]

That feels sub-optimal.

Idea: we could say that r.[x, y] is sugar for [r.x, r.y], then the above would turn into

r.[x, y, z]

Another approach is to have a function in lang.map with some special typing rules

function fieldValues(map<T>, string[] fieldNames) returns T?[];

Technical problem is that we need to get a tuple type with unbroadened types for fieldNames.