[persistence] client pulls of dataframes errors out

[mikeldking]

---------------------------------------------------------------------------
ProtocolError                             Traceback (most recent call last)
File [~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/requests/models.py:816](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/requests/models.py:816), in Response.iter_content.<locals>.generate()
    [815](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/requests/models.py:815) try:
--> [816](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/requests/models.py:816)     yield from self.raw.stream(chunk_size, decode_content=True)
    [817](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/requests/models.py:817) except ProtocolError as e:

File [~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/urllib3/response.py:1040](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/urllib3/response.py:1040), in HTTPResponse.stream(self, amt, decode_content)
   [1039](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/urllib3/response.py:1039) if self.chunked and self.supports_chunked_reads():
-> [1040](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/urllib3/response.py:1040)     yield from self.read_chunked(amt, decode_content=decode_content)
   [1041](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/urllib3/response.py:1041) else:

File [~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/urllib3/response.py:1184](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/urllib3/response.py:1184), in HTTPResponse.read_chunked(self, amt, decode_content)
   [1183](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/urllib3/response.py:1183) while True:
-> [1184](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/urllib3/response.py:1184)     self._update_chunk_length()
   [1185](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/urllib3/response.py:1185)     if self.chunk_left == 0:

File [~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/urllib3/response.py:1119](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/urllib3/response.py:1119), in HTTPResponse._update_chunk_length(self)
   [1117](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/urllib3/response.py:1117) else:
   [1118](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/urllib3/response.py:1118)     # Truncated at start of next chunk
-> [1119](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/urllib3/response.py:1119)     raise ProtocolError("Response ended prematurely") from None

ProtocolError: Response ended prematurely

During handling of the above exception, another exception occurred:
...
--> [818](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/requests/models.py:818)     raise ChunkedEncodingError(e)
    [819](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/requests/models.py:819) except DecodeError as e:
    [820](https://file+.vscode-resource.vscode-cdn.net/Users/mikeldking/work/phoenix/tutorials/tracing/~/anaconda3/envs/phoenix-persist/lib/python3.11/site-packages/requests/models.py:820)     raise ContentDecodingError(e)

ChunkedEncodingError: Response ended prematurely

[axiomofjoy]

For dictionary values, pyarrow requires that all values for a particular key belong to the same type. This is not a guarantee we can provide for user-defined metadata or user-defined attributes more generally. We need to either require that such values are non-dictionaries (e.g., JSON strings) or we need to use a different format for transport, e.g., pickle or JSONL.

[axiomofjoy]

Long-term plan is to move off of pyarrow and return JSON from /spans. Since no users have actually encountered this issue yet, going to punt. If someone hits it, let us know and we will patch the current implementation with pyarrow.

[mikeldking]

Closing as not planned