This registers our own implementation for pyarrow table <-> (Geo)DataFrame conversion through the dispatch mechanism. The distributed p2p shuffle algorithm uses that under the hood (it converts to pyarrow to send the data to other workers), and so with this change (and with the latest distributed release, as we require the fix from https://github.com/dask/distributed/pull/8667), a distributed spatial_shuffle() now works.
This registers our own implementation for pyarrow table <-> (Geo)DataFrame conversion through the dispatch mechanism. The distributed p2p shuffle algorithm uses that under the hood (it converts to pyarrow to send the data to other workers), and so with this change (and with the latest distributed release, as we require the fix from https://github.com/dask/distributed/pull/8667), a distributed
spatial_shuffle()
now works.