search

pathwaycom / pathway

Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
https://pathway.com
Other
2.84k stars 98 forks source link

Allow sending columns with raw bytes in the python connector #39

Open lewymati opened 2 months ago

lewymati commented 2 months ago

Is your feature request related to a problem? Please describe. Cannot stream a table with multiple columns via a Python connector, when one of them is of type bytes. For example:

message = {"col1": "foo", "col2": some_bytes}
self.next_json(message)

doesn't work as bytes are not json serializable. self.next_bytes is not useful as well as it expects bytes encoded json string.

Some workaround is to use base64 to serialize/deserialize, but it's not ideal.