tensorflow / io

Dataset, streaming, and file system extensions maintained by TensorFlow SIG-IO
Apache License 2.0
692 stars 281 forks source link

lint is failing on centos7 #999

Open RuhuaJiang opened 4 years ago

RuhuaJiang commented 4 years ago

How to reproduce:

I am using following docker file:

FROM centos:7

ENV TF_VERSION 2.2.0
ENV IO_VERSION 0.13.0
ENV BZL_VERSION 3.0.0
ENV BZL_URL https://github.com/bazelbuild/bazel/releases/download
ENV BZL_SCRIPT bazel-${BZL_VERSION}-installer-linux-x86_64.sh

RUN yum install -y centos-release-scl
RUN yum install -y curl devtoolset-9 git rh-python36

RUN scl enable rh-python36 devtoolset-9 'python3 -m pip install -U pip'
RUN scl enable rh-python36 devtoolset-9 'python3 -m pip install -U pytest'
RUN scl enable rh-python36 devtoolset-9 'python3 -m pip install -U tensorflow==${TF_VERSION}'

RUN curl -sSOL ${BZL_URL}/${BZL_VERSION}/${BZL_SCRIPT}
RUN bash -x -e ${BZL_SCRIPT}

RUN git clone https://github.com/tensorflow/io
WORKDIR /io
RUN git checkout tags/v${IO_VERSION} -b local-${IO_VERSION}

then build the docker

docker build --tag buildtfio:1.0 .

get into the docker

docker run -v ~/Downloads/tfio_docker:/tfio_docker -it buildtfio:1.0

then run

bazel run //tools/lint:check

we will see

Extracting Bazel installation...
Starting local Bazel server and connecting to it...
WARNING: Download from https://storage.googleapis.com/mirror.tensorflow.org/github.com/bazelbuild/rules_python/archive/38f86fb55b698c51e8510c807489c9f4e047480e.tar.gz failed: class com.google.devtools.build.lib.bazel.repository.downloader.UnrecoverableHttpException GET returned 404 Not Found
INFO: Call stack for the definition of repository 'lint_dependencies' which is a pip_import (rule definition at /root/.cache/bazel/_bazel_root/7e563aa8f1493c3ba33faa85b32d7a3c/external/rules_python/python/pip.bzl:44:14):
 - <builtin>
 - /root/.cache/bazel/_bazel_root/7e563aa8f1493c3ba33faa85b32d7a3c/external/rules_python/python/pip.bzl:114:5
 - /io/WORKSPACE:480:1
ERROR: An error occurred during the fetch of repository 'lint_dependencies':
   pip_import failed:  (src/main/tools/process-wrapper-legacy.cc:58: "execvp(python3, ...)": No such file or directory
)
ERROR: no such package '@lint_dependencies//': pip_import failed:  (src/main/tools/process-wrapper-legacy.cc:58: "execvp(python3, ...)": No such file or directory
)
INFO: Elapsed time: 4.672s
INFO: 0 processes.
yongtang commented 4 years ago

@RuhuaJiang The issue is that CentOS 7's python version is too old and will not work with black or pyupgrade. I think you can use either macOS, Ubuntu 18.04, or Ubuntu 20.04 for lint check.

This is needed for lint check only. You can still use CentOS 7 for binary build.

RuhuaJiang commented 4 years ago

sound good, thanks @yongtang

Shall we still keep this issue open? maybe that's should be a separate issue for addressing Centos 7 python version, i am not sure

yongtang commented 4 years ago

@RuhuaJiang We can leave this issue open for now. I will take a look and see if it is possible to get centos 7 python3 working for lint at a later time.