search

Micromeda / pygenprop

A python library for programmatic usage of EBI InterPro Genome Properties.
http://pygenprop.rtfd.io/
Apache License 2.0
9 stars 4 forks source link

PyArrow's Serialization Functionality is being deprecated. #80

Closed LeeBergstrand closed 2 years ago

LeeBergstrand commented 3 years ago

Problem Description

FutureWarning: 'pyarrow.default_serialization_context' is deprecated as of 2.0.0 and will be removed in a future version. Use pickle or the pyarrow IPC functionality instead.

Recent versions of PyArrow are deprecating the PyArrow serialization functionality. https://arrow.apache.org/docs/python/ipc.html#arbitrary-object-serialization

Early versions of Pygenprop allowed serialization to MSGPACK format via Pandas built into the ToMSGPack function. See https://pandas.pydata.org/pandas-docs/version/0.25.0/reference/api/pandas.DataFrame.to_msgpack.html for details.

This functionality is used by applications such as Micromeda.

Later versions of Pandas drop the to_msgpack functional and recommended that users switch to using PyArrow for serialization. However, PyArrow just announced that they are dropping serialization capability and recommend that users use the latest pickle version instead.

Problems Solution

  1. Go back to using MSGPack using https://github.com/pydata/pandas-msgpack

or

  1. Use pickle. There may be performance concerns with this method.
LeeBergstrand commented 2 years ago

Decided to go the pickle route: https://github.com/Micromeda/pygenprop/pull/81