search

pytorch / torcharrow

High performance model preprocessing library on PyTorch
https://pytorch.org/torcharrow/beta/index.html
BSD 3-Clause "New" or "Revised" License
649 stars 79 forks source link

Fix arrow schema conversion for complex types (#1857) #399

Closed Yuhta closed 2 years ago

Yuhta commented 2 years ago

Summary: X-link: https://github.com/facebookincubator/velox/pull/1857

  1. Use List instead of LargeList because vector_size_t is 32 bits large.
  2. Fix the conversion of Map; arrow requires an extra layer of struct type between the map type and key-value types.

Differential Revision: D37356848

facebook-github-bot commented 2 years ago

This pull request was exported from Phabricator. Differential Revision: D37356848

facebook-github-bot commented 2 years ago

This pull request was exported from Phabricator. Differential Revision: D37356848