search

apache / arrow

Apache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analytics
https://arrow.apache.org/
Apache License 2.0
14.54k stars 3.54k forks source link

[Python] Create Python examples of HTTP GET Arrow client and server using IPC buffer compression #40600

Open ianmcook opened 7 months ago

ianmcook commented 7 months ago

Describe the enhancement requested

Create simple Python-based HTTP client and server examples like the ones in arrow-experiments/http/get_simple/python but using IPC buffer compression (lz4 or zstd).

On the server side, I believe this can be done by passing pa.ipc.IpcWriteOptions(compression=...) as the options argument to pa.ipc.new_stream().

Also see pa.Codec

Component(s)

Python

ianmcook commented 4 months ago

@felipecrv I suggested using Python here for simplicity, but it could be in some other language if needed