search

apache / arrow-rs

Official Rust implementation of Apache Arrow
https://arrow.apache.org/
Apache License 2.0
2.53k stars 759 forks source link

Multi-language support issues with Arrow FlightSQL client's execute_update and execute_ingest methods #6545

Open niebayes opened 3 days ago

niebayes commented 3 days ago

Describe the bug

We designed an Arrow FlightSQL server which implements do_put_statement_update and do_put_statement_ingest. This server works well with the Arrow FlightSQL client of Rust. However, it does not work as expected when interacting with the clients of Go and Python. Specifically, the returned affected rows of the ExecuteUpdate and ExecuteIngest methods is always 0, despite the SQL is executed successfully on the server.

You can find our Go demo codes at https://github.com/niebayes/examples/blob/chore/add_execute_update_examples/go/main.go

You can find our Python demo codes at https://github.com/niebayes/examples/blob/chore/add_execute_update_examples/python/main.py Note, we utilize the flightsql-dbapi library provided by InfluxDB in the Python demo, since Arrow does not provide a native implementation of Arrow FlightSQL yet.

And the Rust demo codes https://github.com/niebayes/examples/blob/chore/add_execute_update_examples/rust/bin/main.rs

To Reproduce

Since our database project is private for now, we cannot provide ways to reproduce the error. But honestly I think this error does not involve the server-side implementation and is solely affected by the client-side deserialization of the DoPutResult.

Expected behavior

We expect the Arrow FlightSQL clients of Go and Python could decode the affected rows correctly, just like the Rust client does.

Additional context

tustvold commented 3 days ago

Tagging @djanderson who added these in https://github.com/apache/arrow-rs/pull/6201

This also sounds a lot like https://github.com/apache/arrow-rs/issues/5731

djanderson commented 3 days ago

Thanks for tagging me, and good timing, I am just wrapping up a concrete implementation of this as well and I have am able to test it with the influx flightsql python client. IIRC I reused an existing codepath that serialized the DoPutResult because it appeared consistent with the spec for do_put_statement_ingest, but https://github.com/apache/arrow-rs/issues/5731 indicates that was not a safe assumption.

I'll try and reproduce this today and get back.