search

tensorflow / addons

Useful extra functionality for TensorFlow 2.x maintained by SIG-addons
Apache License 2.0
1.69k stars 611 forks source link

Build 2.12 #2821

Closed seanpmorgan closed 1 year ago

seanpmorgan commented 1 year ago

Description

Addresses #2819

npanpaliya commented 1 year ago

Hello @seanpmorgan - Any timeline for this PR to get completed and merged into master? We (Open-CE) team is waiting for tensorflow-addons release that works with Tensorflow 2.12.

seanpmorgan commented 1 year ago

Hi @angerson we're seeing:

#23 39.00 ERROR: An error occurred during the fetch of repository 'local_config_cuda':
#23 39.00    Traceback (most recent call last):
#23 39.00   File "/addons/build_deps/toolchains/gpu/cuda_configure.bzl", line 1089, column 38, in _cuda_autoconf_impl
#23 39.00       _create_local_cuda_repository(repository_ctx)
#23 39.00   File "/addons/build_deps/toolchains/gpu/cuda_configure.bzl", line 847, column 35, in _create_local_cuda_repository
#23 39.00       cuda_config = _get_cuda_config(repository_ctx)
#23 39.00   File "/addons/build_deps/toolchains/gpu/cuda_configure.bzl", line 630, column 30, in _get_cuda_config
#23 39.00       config = find_cuda_config(repository_ctx, ["cuda", "cudnn"])
#23 39.00   File "/addons/build_deps/toolchains/gpu/cuda_configure.bzl", line 1079, column 28, in find_cuda_config
#23 39.00       auto_configure_fail("Failed to run find_cuda_config.py: %s" % exec_result.stderr)
#23 39.00   File "/addons/build_deps/toolchains/gpu/cuda_configure.bzl", line 261, column 9, in auto_configure_fail
#23 39.00       fail("\n%sCuda Configuration Error:%s %s\n" % (red, no_color, msg))
#23 39.00 Error in fail: 
#23 39.00 Cuda Configuration Error: Failed to run find_cuda_config.py: Could not find any cuda.h matching version '11.2' in any subdirectory:
#23 39.00         ''
#23 39.00         'include'
#23 39.00         'include/cuda'
#23 39.00         'include/*-linux-gnu'
#23 39.00         'extras/CUPTI/include'
#23 39.00         'include/cuda/CUPTI'
#23 39.00         'local/cuda/extras/CUPTI/include'
#23 39.00 of:
#23 39.00         '/usr/local/cuda'

When building with bazel build --crosstool_top=@ubuntu20.04-gcc9_manylinux2014-cuda11.2-cudnn8.1-tensorrt7.2_config_cuda//crosstool:toolchain

on the new tensorflow/build:2.12-python images. Has there been any changes that you think would cause this?

EDIT -- Missed that it moved to CUDA 11.8

seanpmorgan commented 1 year ago

Hi @kulinseth it looks like tensorflow-macos 2.12 didn't include a python 3.11 release. Was that a mistake or intended? https://pypi.org/project/tensorflow-macos/2.12.0/#files

angerson commented 1 year ago

@seanpmorgan Yep, we moved to 11.8. Let me know if there's anything else that's not working.

kulinseth commented 1 year ago

Hi @kulinseth it looks like tensorflow-macos 2.12 didn't include a python 3.11 release. Was that a mistake or intended? https://pypi.org/project/tensorflow-macos/2.12.0/#files

Hi @seanpmorgan yes , that was intended . We will have 3.11 support from next release onwards .

seanpmorgan commented 1 year ago

Hi @angerson Only the 2.12-python3.10 image is showing:

#24 2.[769](https://github.com/tensorflow/addons/actions/runs/4614368815/jobs/8157266720?pr=2821#step:8:770) ERROR: Exception:
#24 2.769 Traceback (most recent call last):
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/base_command.py", line 160, in exc_logging_wrapper
#24 2.769     status = run_func(*args)
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/cli/req_command.py", line 247, in wrapper
#24 2.769     return func(self, options, args)
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/commands/install.py", line 415, in run
#24 2.769     requirement_set = resolver.resolve(
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/resolution/resolvelib/resolver.py", line 73, in resolve
#24 2.769     collected = self.factory.collect_root_requirements(root_reqs)
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 491, in collect_root_requirements
#24 2.769     req = self._make_requirement_from_install_req(
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 453, in _make_requirement_from_install_req
#24 2.769     cand = self._make_candidate_from_link(
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/resolution/resolvelib/factory.py", line 185, in _make_candidate_from_link
#24 2.769     self._editable_candidate_cache[link] = EditableCandidate(
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 322, in __init__
#24 2.769     super().__init__(
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 162, in __init__
#24 2.769     self.dist = self._prepare()
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 231, in _prepare
#24 2.769     dist = self._prepare_distribution()
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/resolution/resolvelib/candidates.py", line 332, in _prepare_distribution
#24 2.769     return self._factory.preparer.prepare_editable_requirement(self._ireq)
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/operations/prepare.py", line 633, in prepare_editable_requirement
#24 2.769     dist = _get_prepared_distribution(
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/operations/prepare.py", line 69, in _get_prepared_distribution
#24 2.769     abstract_dist.prepare_distribution_metadata(
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/distributions/sdist.py", line 46, in prepare_distribution_metadata
#24 2.769     self.req.isolated_editable_sanity_check()
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/req/req_install.py", line 505, in isolated_editable_sanity_check
#24 2.769     and not self.supports_pyproject_editable()
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_internal/req/req_install.py", line 245, in supports_pyproject_editable
#24 2.769     return "build_editable" in self.pep517_backend._supported_features()
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_vendor/pyproject_hooks/_impl.py", line 153, in _supported_features
#24 2.769     return self._call_hook('_supported_features', {})
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_vendor/pyproject_hooks/_impl.py", line 321, in _call_hook
#24 2.769     raise BackendUnavailable(data.get('traceback', ''))
#24 2.769 pip._vendor.pyproject_hooks._impl.BackendUnavailable: Traceback (most recent call last):
#24 2.769   File "/usr/local/lib/python3.10/dist-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 77, in _build_backend
#24 2.769     obj = import_module(mod_path)
#24 2.769   File "/usr/lib/python3.10/importlib/__init__.py", line 126, in import_module
#24 2.769     return _bootstrap._gcd_import(name[level:], package, level)
#24 2.769   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
#24 2.769   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
#24 2.769   File "<frozen importlib._bootstrap>", line 992, in _find_and_load_unlocked
#24 2.769   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
#24 2.769   File "<frozen importlib._bootstrap>", line 1050, in _gcd_import
#24 2.769   File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
#24 2.769   File "<frozen importlib._bootstrap>", line 1004, in _find_and_load_unlocked
#24 2.769 ModuleNotFoundError: No module named 'setuptools'

Was something missed in the build? <3.10 and 3.11 work fine. We can hard code a fix but would rather it just be fixed in the image if possible

seanpmorgan commented 1 year ago

Was something missed in the build? <3.10 and 3.11 work fine. We can hard code a fix but would rather it just be fixed in the image if possible

Looks like its related to https://github.com/pypa/pip/issues/11770 . We will hardcode a pip version for now

bhack commented 1 year ago

@seanpmorgan I've fixed the last few issues, green lights now.

angerson commented 1 year ago

Looks like its related to https://github.com/pypa/pip/issues/11770 . We will hardcode a pip version for now

Makes sense. Around Python 3.10 when distutils (?) was dropped, a lot of stuff broke. Good to hear there's a workaround.