Enable building TorchArrow with PyTorch in Facebook CI

pytorch / torcharrow

High performance model preprocessing library on PyTorch

https://pytorch.org/torcharrow/beta/index.html

BSD 3-Clause "New" or "Revised" License

649 stars 79 forks source link

Enable building TorchArrow with PyTorch in Facebook CI #334

Closed Nayef211 closed 2 years ago

Nayef211 commented 2 years ago

This PR enables building TorchArrow with PyTorch in CI. This also allows us to test the functionality of some of the torchtext operators that were added in https://github.com/facebookresearch/torcharrow/pull/271 and https://github.com/facebookresearch/torcharrow/pull/287

facebook-github-bot commented 2 years ago

@Nayef211 has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

Nayef211 commented 2 years ago

@wenleix any suggestions on fixing the CI error? From the error message, the build thinks we're using CUDA even though I'm explicitly installing the cpu version of torch.

CMake Error at /home/runner/.local/lib/python3.7/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:90 (message):
  Your installed Caffe2 version uses CUDA but I cannot find the CUDA
  libraries.  Please set the proper CUDA prefixes and / or install CUDA.
Call Stack (most recent call first):
  /home/runner/.local/lib/python3.7/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  csrc/velox/CMakeLists.txt:73 (find_package)

cc @mthrok

mthrok commented 2 years ago

@wenleix any suggestions on fixing the CI error? From the error message, the build thinks we're using CUDA even though I'm explicitly installing the cpu version of torch.

CMake Error at /home/runner/.local/lib/python3.7/site-packages/torch/share/cmake/Caffe2/Caffe2Config.cmake:90 (message):
  Your installed Caffe2 version uses CUDA but I cannot find the CUDA
  libraries.  Please set the proper CUDA prefixes and / or install CUDA.
Call Stack (most recent call first):
  /home/runner/.local/lib/python3.7/site-packages/torch/share/cmake/Torch/TorchConfig.cmake:68 (find_package)
  csrc/velox/CMakeLists.txt:73 (find_package)

cc @mthrok

The change list looks okay, and the CI log indeed says it is installing CPU version. And I expect this kind of issue from CUDA-enabled torch installation. So I am not sure why.

facebook-github-bot commented 2 years ago

This pull request was exported from Phabricator. Differential Revision: D36481571

facebook-github-bot commented 2 years ago

This pull request was exported from Phabricator. Differential Revision: D36481571

facebook-github-bot commented 2 years ago

This pull request was exported from Phabricator. Differential Revision: D36481571