Building with prebuilt_python_library fails with "no suitable image found. Did find"

[Nikita2k]

Hey!

I'm trying to run a simple python script with pytorch. Here is what I have:

# client.py
import torch

if __name__ == '__main__':
  N, D_in = 64, 1000
  t = torch.randn(N, D_in)
  print(f"t: {t}")

BUCK

python_binary(
  name = 'client',
  base_module = '',
  main_module = 'client',
  deps = [
    ':client_lib',
  ],
)

python_library(
  name = 'client_lib',
  base_module = '',
  srcs = [
    'client.py'
  ],
  deps = [
    '//external:pytorch_whl',
  ]
)

prebuilt_python_library(
  name = 'pytorch_whl',
  binary_src = 'torch-1.7.1-cp39-none-macosx_10_9_x86_64.whl',
  visibility = [
    'PUBLIC',
  ],
)

I got a wheel from pytorch.org, as well as I tried building it from source. Whenever I try running with buck run //client/py:client it fails with:

Not using buckd because watchman isn't installed.
Parsing buck files: finished in 2.1 sec
Building: finished in 39.7 sec (100%) 3/3 jobs, 3 updated
  Total time: 42.0 sec
Traceback (most recent call last):
  File ".bootstrap/_pex/pex.py", line 328, in execute
  File ".bootstrap/_pex/pex.py", line 261, in _wrap_coverage
  File ".bootstrap/_pex/pex.py", line 293, in _wrap_profiling
  File ".bootstrap/_pex/pex.py", line 371, in _execute
  File ".bootstrap/_pex/pex.py", line 429, in execute_entry
  File ".bootstrap/_pex/pex.py", line 434, in execute_module
  File "/Users/user_name/.pyenv/versions/3.8.6/lib/python3.8/runpy.py", line 210, in run_module
    return _run_code(code, {}, init_globals, run_name, mod_spec)
  File "/Users/user_name/.pyenv/versions/3.8.6/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "client.py", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 655, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 618, in _load_backward_compatible
  File "<frozen zipimport>", line 259, in load_module
  File "torch/__init__.py", line 189, in <module>
  File "torch/__init__.py", line 142, in _load_global_deps
  File "/Users/user_name/.pyenv/versions/3.8.6/lib/python3.8/ctypes/__init__.py", line 373, in __init__
    self._handle = _dlopen(self._name, mode)
OSError: dlopen(/Users/user_name/proj_name/buck-out/gen/client/py/client.pex/torch/lib/libtorch_global_deps.dylib, 10): no suitable image found.  Did find:
    /Users/user_name/proj_name/buck-out/gen/client/py/client.pex/torch/lib/libtorch_global_deps.dylib: stat() failed with errno=20

I tried searching for similar errors but had no luck. Am I missing something?