RayDP Docker Image with Python 3.10 Failing to Build

[dattasoumya92]

Issue: We are trying to build the docker image for RayDP from the dockerfile given in this project Dockerfile from here, and instead of the default base image ray:latest we are trying to use the python 3.10 version of Ray base image i.e. ray:latest-py310.

However when building the docker image we are getting below build error: Seems there is an issue with installation of netifaces pip package.

Downloading raydp-1.6.0-py3-none-any.whl (14.4 MB)
   ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 14.4/14.4 MB 35.1 MB/s eta 0:00:00
Building wheels for collected packages: pyspark, netifaces
  Building wheel for pyspark (setup.py): started
  Building wheel for pyspark (setup.py): finished with status 'done'
  Created wheel for pyspark: filename=pyspark-3.3.2-py2.py3-none-any.whl size=281824009 sha256=165fdbc20c4e94d4f4dc97e767ce32dc5f082087f66254de49e59ddb54bdf01c
  Stored in directory: /tmp/pip-ephem-wheel-cache-rbl50jik/wheels/89/d6/52/1178e354ba2207673484f0ccd7b2ded0ab6671ae5c1fc5b49a
  Building wheel for netifaces (setup.py): started
  Building wheel for netifaces (setup.py): finished with status 'error'
  error: subprocess-exited-with-error

  × python setup.py bdist_wheel did not run successfully.
  │ exit code: 1
  ╰─> [29 lines of output]
      /home/ray/anaconda3/lib/python3.10/site-packages/setuptools/config/setupcfg.py:293: _DeprecatedConfig: Deprecated config in `setup.cfg`
      !!

              ********************************************************************************
              The license_file parameter is deprecated, use license_files instead.

              This deprecation is overdue, please update your project and remove deprecated
              calls to avoid build errors in the future.

              See https://setuptools.pypa.io/en/latest/userguide/declarative_config.html for details.
              ********************************************************************************

      !!
        parsed = self.parsers.get(option_name, lambda x: x)(value)
      running bdist_wheel
      running build
      running build_ext
      checking for getifaddrs...not found.
      checking for getnameinfo...not found.
      checking for socket IOCTLs...not found.
      checking for optional header files...none found.
      checking whether struct sockaddr has a length field...no.
      checking which sockaddr_xxx structs are defined...none!
      checking for routing socket support...no.
      checking for sysctl(CTL_NET...) support...no.
      checking for netlink support...no.
      building 'netifaces' extension
      gcc -pthread -B /home/ray/anaconda3/compiler_compat -Wno-unused-result -Wsign-compare -DNDEBUG -fwrapv -O2 -Wall -fPIC -O2 -isystem /home/ray/anaconda3/include -fPIC -O2 -isystem /home/ray/anaconda3/include -fPIC -DNETIFACES_VERSION=0.11.0 -I/home/ray/anaconda3/include/python3.10 -c netifaces.c -o build/temp.linux-x86_64-cpython-310/netifaces.o
      error: command 'gcc' failed: No such file or directory
      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
  ERROR: Failed building wheel for netifaces
  Running setup.py clean for netifaces

Expected Behavior: Docker Image to build successfully.

Actual Behavior: Image build is failing.

Note: We have tried installing raydp on an amazon linux 2 VM with python 3.10 (official python not Anaconda) and it got installed fine and worked as expected.

[kira-lin]

it says gcc is missing. Maybe you can try apt install build-essential before installing netifaces in dockerfile and uninstall it in the end.

[dattasoumya92]

@kira-lin Thanks for quick response. After adding apt install build-essential my docker build is successful. Would be better if this be added in the Dockerfile in this project or at-least mentioned in the README.md. Thanks again.