tensorboard: build issues on Apple M1 (arm64)

[adamjstewart]

Environment information (required)

Diagnostics output

`````` --- check: autoidentify INFO: diagnose_tensorboard.py version e43767ef2b648d0d5d57c00f38ccbd38390e38da --- check: general INFO: sys.version_info: sys.version_info(major=3, minor=8, micro=9, releaselevel='final', serial=0) INFO: os.name: posix INFO: os.uname(): posix.uname_result(sysname='Darwin', nodename='ajstewart-mbp', release='21.4.0', version='Darwin Kernel Version 21.4.0: Fri Mar 18 00:46:32 PDT 2022; root:xnu-8020.101.4~15/RELEASE_ARM64_T6000', machine='arm64') INFO: sys.getwindowsversion(): N/A --- check: package_management INFO: has conda-meta: False INFO: $VIRTUAL_ENV: None --- check: installed_packages WARNING: no installation among: ['tb-nightly', 'tensorboard', 'tensorflow-tensorboard'] WARNING: no installation among: ['tensorflow', 'tensorflow-gpu', 'tf-nightly', 'tf-nightly-2.0-preview', 'tf-nightly-gpu', 'tf-nightly-gpu-2.0-preview'] WARNING: no installation among: ['tensorflow-estimator', 'tensorflow-estimator-2.0-preview', 'tf-estimator-nightly'] --- check: tensorboard_python_version Traceback (most recent call last): File "diagnose_tensorboard.py", line 522, in main suggestions.extend(check()) File "diagnose_tensorboard.py", line 75, in wrapper result = fn() File "diagnose_tensorboard.py", line 271, in tensorboard_python_version from tensorboard import version ModuleNotFoundError: No module named 'tensorboard' --- check: tensorflow_python_version Traceback (most recent call last): File "diagnose_tensorboard.py", line 522, in main suggestions.extend(check()) File "diagnose_tensorboard.py", line 75, in wrapper result = fn() File "diagnose_tensorboard.py", line 278, in tensorflow_python_version import tensorflow as tf ModuleNotFoundError: No module named 'tensorflow' --- check: tensorboard_data_server_version INFO: no data server installed --- check: tensorboard_binary_path INFO: which tensorboard: None --- check: addrinfos socket.has_ipv6 = True socket.AF_UNSPEC = socket.SOCK_STREAM = socket.AI_ADDRCONFIG = socket.AI_PASSIVE = Loopback flags: Loopback infos: [(, , 6, '', ('::1', 0, 0, 0)), (, , 6, '', ('127.0.0.1', 0))] Wildcard flags: Wildcard infos: [(, , 6, '', ('::', 0, 0, 0)), (, , 6, '', ('0.0.0.0', 0))] --- check: readable_fqdn INFO: socket.getfqdn(): 'ajstewart-mbp.dhcp.thefacebook.com' --- check: stat_tensorboardinfo INFO: directory: /var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/.tensorboard-info INFO: .tensorboard-info directory does not exist --- check: source_trees_without_genfiles INFO: tensorboard_roots (0): []; bad_roots (0): [] --- check: full_pip_freeze INFO: pip freeze --all: pip==20.2.3 setuptools==49.2.1 six @ file:///AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/python3/python3-110/six-1.15.0-py2.py3-none-any.whl wheel @ file:///AppleInternal/BuildRoot/Library/Caches/com.apple.xbs/Sources/python3/python3-110/wheel-0.36.2-py2.py3-none-any.whl ``````

Steps to reproduce (required)

I'm trying to build tensorboard from source using the Spack package manager. The build steps basically involve:

$ bazel build //tensorboard/pip_package

I see an issue that suggests that the build doesn't work for arm64:

ERROR: An error occurred during the fetch of repository 'npm':
   Traceback (most recent call last):
        File "/private/var/folders/j1/68dlgpr91vlgs26vty2c8xk80000gn/T/spack01rlj_pt/fe82cc522352db2e5566862b12b0534f/external/build_bazel_rules_nodejs/internal/npm_install/npm_install.bzl", line 823, column 31, in _yarn_install_impl
                node = repository_ctx.path(get_node_label(repository_ctx))
Error in path: Unable to load package for @nodejs_darwin_arm64//:bin/node: The repository '@nodejs_darwin_arm64' could not be resolved: Repository '@nodejs_darwin_arm64' is not defined

Is there any way to build from source on arm64 or is the only way to install from a pre-compiled wheel?

Related to #5713

@glennpj @aweits

[adamjstewart]

@japie1235813 are there any plans to add Apple M1 build support to tensorboard? We got tensorboard-data-server working in #5715 but this is a bit more complicated.

[nfelt]

Could you elaborate? The tensorboard pip package itself is pure python, so I'm not aware of any platform incompatibility with M1 that would be inherent in the built package itself.

If you instead mean that some part of the build toolchain or some dependency (like tensorflow) doesn't support M1, it's relatively unlikely we'll be able to easily address that, although if it's not a major change we'd probably be willing to accept a contribution.

[adamjstewart]

To clarify, I'm not using pip, I'm using the Spack package manager. In Spack, we try to build everything from source instead of relying on pre-packaged binaries like wheels. Unfortunately, our build recipe for tensorboard doesn't seem to work on M1. If we have to, we can install from wheels, but we would prefer to build from source if possible.

[adamjstewart]

I dug a bit deeper into this. According to https://github.com/bazelbuild/rules_nodejs/issues/3101, node-js 14 didn't have rules for darwin_arm64, you have to use node-js 16 instead. This was fixed in https://github.com/bazelbuild/rules_nodejs/pull/3107 which upgrades the version of node-js used by rules_nodejs. So to solve this specific issue (there may be more later), we need to upgrade to rules_nodejs 5+.

I tried replacing:

http_archive(
    name = "build_bazel_rules_nodejs",
    sha256 = "d63ecec7192394f5cc4ad95a115f8a6c9de55c60d56c1f08da79c306355e4654",
    urls = [
        "http://mirror.tensorflow.org/github.com/bazelbuild/rules_nodejs/releases/download/4.6.1/rules_nodejs-4.6.1.tar.gz",
        "https://github.com/bazelbuild/rules_nodejs/releases/download/4.6.1/rules_nodejs-4.6.1.tar.gz",
    ],
)

with:

http_archive(
    name = "build_bazel_rules_nodejs",
    sha256 = "ee3280a7f58aa5c1caa45cb9e08cbb8f4d74300848c508374daf37314d5390d6",
    urls = ["https://github.com/bazelbuild/rules_nodejs/releases/download/5.5.1/rules_nodejs-5.5.1.tar.gz"],
)

in WORKSPACE but I encounter the following error when I try to build:

ERROR: Failed to load Starlark extension '@rules_nodejs//nodejs:providers.bzl'.
Cycle in the workspace file detected. This indicates that a repository is used prior to being defined.
The following chain of repository dependencies lead to the missing definition.
 - @rules_nodejs
This could either mean you have to add the '@rules_nodejs' repository with a statement like `http_archive` in your WORKSPACE file (note that transitive dependencies are not added automatically), or move an existing definition earlier in your WORKSPACE file.

I don't know bazel well enough to debug this. The last time rules_nodejs was updated was by @JamesHollyer in #5659.

[nfelt]

To clarify, I'm not using pip, I'm using the Spack package manager. In Spack, we try to build everything from source instead of relying on pre-packaged binaries like wheels. Unfortunately, our build recipe for tensorboard doesn't seem to work on M1. If we have to, we can install from wheels, but we would prefer to build from source if possible.

FWIW, I am not familiar with Spack, but the build recipe pretty clearly indicates that you're invoking our pip package build script and calling pip at the end to install it, so I stand by my point that the artifact you're trying to build is not platform-specific (and hence whatever M1 incompatibility exists in the build process, it's limited purely to the process, not the end result).

You're welcome to file a feature request for us to update to rules_nodejs 5+, but that's not a trivial upgrade process and there's no guarantee at all in terms of the timing of when we would do that upgrade, so I would suggest using the prebuilt wheel to get unblocked.

[adamjstewart]

Thanks! I'll use the pre-built wheel for now. Consider this issue a request to upgrade to rules_nodejs 5+ when you get a chance. Hopefully that's all that's needed for native development builds on Apple M1. For example, if I want to build directly from the master branch or want to test a change I've made to the source code, the pre-build wheels for each release may not be sufficient.