tensorboard 2.5.0 requires bazel 3.7+, but tensorboard-plugin-wit doesn't build with bazel 3.7+

adamjstewart commented 3 years ago

I'm trying to update our Spack recipe for TensorBoard to add 2.5.0. However, TensorBoard 2.5.0 requires Bazel 3.7+, but tensorboard-plugin-wit (a dependency) doesn't build with Bazel 3.7+. See the following issues for details:

If tensorboard-plugin-wit can't be compiled with Bazel 3.7+, and if TensorBoard requires both tensorboard-plugin-wit and Bazel 3.7+, then how can I install tensorboard 2.5.0?

Environment information (required)

Diagnostics

Diagnostics output

`````` --- check: autoidentify INFO: diagnose_tensorboard.py version e43767ef2b648d0d5d57c00f38ccbd38390e38da --- check: general INFO: sys.version_info: sys.version_info(major=3, minor=8, micro=10, releaselevel='final', serial=0) INFO: os.name: posix INFO: os.uname(): posix.uname_result(sysname='Linux', nodename='LAPTOP-VG0BM82I', release='5.10.16.3-microsoft-standard-WSL2', version='#1 SMP Fri Apr 2 22:23:49 UTC 2021', machine='x86_64') INFO: sys.getwindowsversion(): N/A --- check: package_management INFO: has conda-meta: False INFO: $VIRTUAL_ENV: None --- check: installed_packages /usr/bin/python: No module named pip Traceback (most recent call last): File "diagnose_tensorboard.py", line 522, in main suggestions.extend(check()) File "diagnose_tensorboard.py", line 179, in installed_packages freeze = pip(["freeze", "--all"]).decode("utf-8").splitlines() File "diagnose_tensorboard.py", line 103, in pip return subprocess.check_output(command) File "/usr/lib/python3.8/subprocess.py", line 415, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/usr/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/python', '-m', 'pip', '--disable-pip-version-check', 'freeze', '--all']' returned non-zero exit status 1. --- check: tensorboard_python_version INFO: tensorboard.version.VERSION: '2.4.1' --- check: tensorflow_python_version Traceback (most recent call last): File "diagnose_tensorboard.py", line 522, in main suggestions.extend(check()) File "diagnose_tensorboard.py", line 75, in wrapper result = fn() File "diagnose_tensorboard.py", line 278, in tensorflow_python_version import tensorflow as tf ModuleNotFoundError: No module named 'tensorflow' --- check: tensorboard_data_server_version INFO: no data server installed --- check: tensorboard_binary_path INFO: which tensorboard: None --- check: addrinfos socket.has_ipv6 = True socket.AF_UNSPEC = socket.SOCK_STREAM = socket.AI_ADDRCONFIG = socket.AI_PASSIVE = Loopback flags: Loopback infos: [(, , 6, '', ('::1', 0, 0, 0)), (, , 6, '', ('127.0.0.1', 0))] Wildcard flags: Wildcard infos: [(, , 6, '', ('0.0.0.0', 0)), (, , 6, '', ('::', 0, 0, 0))] --- check: readable_fqdn INFO: socket.getfqdn(): 'LAPTOP-VG0BM82I.localdomain' --- check: stat_tensorboardinfo INFO: directory: /tmp/.tensorboard-info INFO: .tensorboard-info directory does not exist --- check: source_trees_without_genfiles INFO: tensorboard_roots (1): ['/home/t-astewart/spack/opt/spack/linux-ubuntu20.04-skylake/gcc-9.3.0/py-tensorboard-2.4.1-5rt67paz4lxdffj74ynsiux43bpe2ner/lib/python3.8/site-packages']; bad_roots (0): [] --- check: full_pip_freeze /usr/bin/python: No module named pip Traceback (most recent call last): File "diagnose_tensorboard.py", line 522, in main suggestions.extend(check()) File "diagnose_tensorboard.py", line 75, in wrapper result = fn() File "diagnose_tensorboard.py", line 489, in full_pip_freeze "pip freeze --all:\n%s", pip(["freeze", "--all"]).decode("utf-8") File "diagnose_tensorboard.py", line 103, in pip return subprocess.check_output(command) File "/usr/lib/python3.8/subprocess.py", line 415, in check_output return run(*popenargs, stdout=PIPE, timeout=timeout, check=True, File "/usr/lib/python3.8/subprocess.py", line 516, in run raise CalledProcessError(retcode, process.args, subprocess.CalledProcessError: Command '['/usr/bin/python', '-m', 'pip', '--disable-pip-version-check', 'freeze', '--all']' returned non-zero exit status 1. ``````

Next steps

No action items identified. Please copy ALL of the above output, including the lines containing only backticks, into your GitHub issue or comment. Be sure to redact any sensitive information.

Steps to reproduce (required)

$ git clone https://github.com/spack/spack.git
$ . spack/share/spack/setup-env.sh
$ spack install py-tensorboard@2.5.0 ^bazel@3.7:

japie1235813 commented 3 years ago

Normally users use prebuilt version of deps even building TensorBoard from source. We cannot guarantee the bazel version our deps get built. As it is discussed in https://github.com/PAIR-code/what-if-tool/issues/167, you can A) convince tb-plugin-wit to support 3.7+ B) use a prebuilt version of tb-plugin-wit that used old Bazel C) omit it entirely; tensorboard just won't have the What-If Tool plugin available then

Close this issue. Feel free to reopen if you have more questions.

adamjstewart commented 3 years ago

Thanks, I didn't realize tb-plugin-wit was optional, which features does it provide? If I can omit that then that will simplify the build. (@aweits we should look into this)

nfelt commented 3 years ago

The tensorboard-plugin-wit package provides a dynamic plugin containing the What-If Tool (described more at https://github.com/PAIR-code/what-if-tool and https://www.tensorflow.org/tensorboard/what_if_tool). Not having it available won't affect the rest of the functionality of TensorBoard, but that specific dashboard will not be available in the UI.

adamjstewart commented 3 years ago

Based on https://github.com/tensorflow/tensorboard/blob/master/tensorboard/pip_package/requirements.txt#L28, it doesn't seem like tensorboard-plugin-wit is optional. Shouldn't it be in extra_requires instead of install_requires?

nfelt commented 3 years ago

It's included as a required dependency since the dashboard is meant to be present by default whenever TensorBoard is installed, and putting it in extra_requires would require most users to take an extra step to get the dashboard. Or to put it another way, our position is that it's required for the complete feature set that TensorBoard is documented to provide, even if TensorBoard functions without it.

Also, this generally hasn't been a problematic dependency for users since they just use the wheel, so there hasn't been much of a reason to unbundle it. I'm not very familiar with Spack, but I'd be surprised if there was anything about that particular pip package that benefited materially from being rebuilt from source in order to optimize for supercomputers.

tensorflow / tensorboard