We are making heavy use of pd.Categorical. Unfortunately Chunkstore does not support this at the moment. One very simple way to change this would be to change FrameConverter to use pickle to serialize columns instead of arr.tostring(). See link to my fork below. The fork also removes pd.Panel imports that are incompatible with pandas 1.2.
Is this a feature of general interest? If so, how should a pull request for this look like? I assume this change should be optional to prevent breaking existing code.
adxrgh
commented
3 years ago
We are making heavy use of pd.Categorical. Unfortunately
Chunkstoredoes not support this at the moment. One very simple way to change this would be to changeFrameConverterto use pickle to serialize columns instead of arr.tostring(). See link to my fork below. The fork also removes pd.Panel imports that are incompatible with pandas 1.2.Is this a feature of general interest? If so, how should a pull request for this look like? I assume this change should be optional to prevent breaking existing code.
https://github.com/bjonen/arctic/commit/f44d194daf44c01a29c95c0067d74ebc2678546a