Closed dariogoetz closed 6 months ago
I'm a little late to this reply given the WIP pr, but I was studying some PyO3 myself :). In this case, not 100% sure, but I'm wondering if defining an IntoPyProduceOutput
IntoPy<RecordMeta>
would work out. Not sure if that works with the async ProduceOutput
wait()
.
It seems like these sections discussing IntoPy
seem interesting for this case:
https://pyo3.rs/main/doc/pyo3/conversion/trait.intopy#conversion-to-a-python-object
https://pyo3.rs/main/doc/pyo3/conversion/trait.intopy#dynamic-conversion-into-python-objects
I have a use-case in which a fluvio producer needs to keep track of the offsets (and partitions) that it sent its messages to. To that end, I would like to expose the
RecordMetadata
struct to the python client.If I understand correctly, the
TopicProducer.send
method (in rust) generates aProduceOutput
struct, which provides await
method in the end yielding the desiredRecordMetadata
.I see two general ways forward to expose such functionality in the python client:
ProduceOutput
and aRecordMetadata
to the python world, effectively mirroring the rust variants. This has a weird caveat, though: TheProduceOutput.wait
method consumesself
, inhibiting any further use of theProduceOutput
object. If aProduceOutput
would also live in the python world, such a "consumption" is not possible to have (if I understand PyO3 correctly). This could be worked around by wrapping the "inner" rust-version of theProduceOutput
in aCell
and then using
take
in the correspondingwait
method:ProduceOutput
and directly provide some methodsend_and_wait_for_metadata
(a proper name would need to be found) that callswait
on the result itself and then returns aRecordMetadata
object.Do you have a good idea, how to address this issue?
I would be willing to add a PR for either of the approaches (or another one, if you have an idea).