tradewelltech / protarrow

Convert from protobuf to arrow and back
https://protarrow.readthedocs.io/
Apache License 2.0
17 stars 2 forks source link

Field presence in non-message type fields #68

Closed chasezheng closed 6 months ago

chasezheng commented 7 months ago

I really love this library. It's very handy.

For proto3, it's possible for check for presence of a string field. For example,

message MyMsg {
  optional string value= 1;
}

And then we can check

MyMsg.DESCRIPTOR.fields[0].has_presence  # is true

MyMsg().HasField('value')   # is false

However, when converting the proto to arrow and then back. The presence bit is not preserved. It would be great if you can change this.

0x26res commented 6 months ago

I think when we started working on protobuf the optional label wasn't working as intended.

So instead we used wrapped primitive, like DoubleValue, Int32Value etc.

I had a look and the current version of protobuf optional behave similarly to the wrapped primitive, with less nesting (which is a better API). So we should add support for it.

0x26res commented 6 months ago

@chasezheng this is fixed in v0.4.0. Please give it a try.