search

ray-project / pygloo

Pygloo provides Python bindings for Gloo.
Apache License 2.0
16 stars 9 forks source link

Problem compiling the Pygloo with Python 3.9 #8

Closed aavbsouza closed 3 years ago

aavbsouza commented 3 years ago

Hello everyone, I am having some issues to compile the library using python 3.9 in a ubuntu 20.0.4 docker environment I could compile the library (only changing the supported python versions) on a fedora host. But inside the container I get this error

PATH=/usr/local/bazel/bin:$PATH python setup.py  install
running install
running bdist_egg
running egg_info
writing pygloo.egg-info/PKG-INFO
writing dependency_links to pygloo.egg-info/dependency_links.txt
writing top-level names to pygloo.egg-info/top_level.txt
reading manifest file 'pygloo.egg-info/SOURCES.txt'
writing manifest file 'pygloo.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_ext
Starting local Bazel server and connecting to it...
DEBUG: /root/.cache/bazel/_bazel_root/e9abd5bf1f434ac0d0a89ac58b0b9aa1/external/rules_foreign_cc/workspace_definitions.bzl:15:10: WARNING: This branch is deprecated and no longer recieving updates. Please update to `main` or choose a specific commit to pin in your workspace.
INFO: Analyzed 5 targets (24 packages loaded, 864 targets configured).
INFO: Found 5 targets...
ERROR: /pygloo/pygloo/BUILD:6:15: output 'pygloo/libuv/lib64/libuv_a.a' was not created
ERROR: /pygloo/pygloo/BUILD:6:15: not all outputs were created or valid
INFO: Elapsed time: 35.556s, Critical Path: 33.15s
INFO: 2 processes: 1 internal, 1 processwrapper-sandbox.
FAILED: Build did NOT complete successfully
Traceback (most recent call last):
  File "/pygloo/setup.py", line 131, in <module>
    setuptools.setup(
  File "/usr/local/python/lib/python3.9/site-packages/setuptools/__init__.py", line 153, in setup
    return distutils.core.setup(**attrs)
  File "/usr/local/python/lib/python3.9/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/local/python/lib/python3.9/distutils/dist.py", line 966, in run_commands
    self.run_command(cmd)
  File "/usr/local/python/lib/python3.9/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/local/python/lib/python3.9/site-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/usr/local/python/lib/python3.9/site-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/usr/local/python/lib/python3.9/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/local/python/lib/python3.9/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/local/python/lib/python3.9/site-packages/setuptools/command/bdist_egg.py", line 164, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/usr/local/python/lib/python3.9/site-packages/setuptools/command/bdist_egg.py", line 150, in call_command
    self.run_command(cmdname)
  File "/usr/local/python/lib/python3.9/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/local/python/lib/python3.9/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/usr/local/python/lib/python3.9/site-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/usr/local/python/lib/python3.9/distutils/command/install_lib.py", line 107, in build
    self.run_command('build_ext')
  File "/usr/local/python/lib/python3.9/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/local/python/lib/python3.9/distutils/dist.py", line 985, in run_command
    cmd_obj.run()
  File "/pygloo/setup.py", line 116, in run
    return pip_run(self)
  File "/pygloo/setup.py", line 101, in pip_run
    build()
  File "/pygloo/setup.py", line 94, in build
    return bazel_invoke(
  File "/pygloo/setup.py", line 48, in bazel_invoke
    result = invoker([cmd] + cmdline, *args, **kwargs)
  File "/usr/local/python/lib/python3.9/subprocess.py", line 373, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bazel', 'build', '--verbose_failures', '--', '//pygloo:all']' returned non-zero exit status 1.

On this container environment, outside the bazel build system. I could compile both the hiredis and libuv libraries without any problems.

Thanks

zhisbug commented 3 years ago

@aavbsouza Thanks for submitting the issue. @Ezra-H Could you look into it?

huangrh99 commented 3 years ago

Hi, @aavbsouza. You need to change the library directory of libuv in pygloo/BUILD from lib64 to lib. So, there are two places you need to modify.

  1. out_lib_dir = "lib64", to out_lib_dir = "lib",
  2. "libuv_LIBDIR": "$EXT_BUILD_DEPS/libuv/lib64", to "libuv_LIBDIR": "$EXT_BUILD_DEPS/libuv/lib",
aavbsouza commented 3 years ago

Hello @Ezra-H , thanks for you reply, this workaround solved the issue

jovany-wang commented 2 years ago

The same problem is current CI:

INFO: Found 5 targets...
[0 / 3] [Prepa] Writing script pygloo/gloo_scripts/CMake_script.sh
[4 / 7] CcCmakeMakeRule pygloo/libuv/include; 1s processwrapper-sandbox ... (2 actions running)
[14 / 32] CcCmakeMakeRule pygloo/libuv/include; 3s processwrapper-sandbox
[14 / 32] CcCmakeMakeRule pygloo/libuv/include; 5s processwrapper-sandbox
[14 / 32] CcCmakeMakeRule pygloo/libuv/include; 14s processwrapper-sandbox
[14 / 32] CcCmakeMakeRule pygloo/libuv/include; 36s processwrapper-sandbox
[14 / 32] CcCmakeMakeRule pygloo/libuv/include; 44s processwrapper-sandbox
ERROR: /__w/pygloo/pygloo/pygloo/BUILD:6:15: output 'pygloo/libuv/lib64/libuv_a.a' was not created
ERROR: /__w/pygloo/pygloo/pygloo/BUILD:6:15: CcCmakeMakeRule pygloo/libuv/include failed: not all outputs were created or valid
INFO: Elapsed time: 57.6[24](https://github.com/ray-project/pygloo/runs/5861657429?check_suite_focus=true#step:6:24)s, Critical Path: 45.90s
INFO: 15 processes: 12 internal, 3 processwrapper-sandbox.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully
/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
Traceback (most recent call last):
  File "setup.py", line 1[31](https://github.com/ray-project/pygloo/runs/5861657429?check_suite_focus=true#step:6:31), in <module>
    setuptools.setup(
  File "/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/__init__.py", line 77, in setup
    return distutils.core.setup(**attrs)
  File "/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 148, in setup
    return run_commands(dist)
  File "/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/_distutils/core.py", line 163, in run_commands
    dist.run_commands()
  File "/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 967, in run_commands
    self.run_command(cmd)
  File "/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/dist.py", line 1196, in run_command
    super().run_command(command)
  File "/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
    cmd_obj.run()
  File "/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/command/install.py", line 72, in run
    orig.install.run(self)
  File "/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/_distutils/command/install.py", line 670, in run
    self.run_command('build')
  File "/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/dist.py", line 1196, in run_command
    super().run_command(command)
  File "/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 986, in run_command
    cmd_obj.run()
  File "/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/_distutils/command/build.py", line 1[35](https://github.com/ray-project/pygloo/runs/5861657429?check_suite_focus=true#step:6:35), in run
    self.run_command(cmd_name)
  File "/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/_distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/dist.py", line 1196, in run_command
    super().run_command(command)
  File "/__w/pygloo/pygloo/py3/lib/python3.8/site-packages/setuptools/_distutils/dist.py", line 986, in run_command