Cannot view example plugins in tensorboard

[meltzerpete]

Environment information (required)

Diagnostics

Diagnostics output

`````` --- check: autoidentify INFO: diagnose_tensorboard.py version 516a2f9433ba4f9c3a4fdb0f89735870eda054a1 --- check: general INFO: sys.version_info: sys.version_info(major=3, minor=9, micro=13, releaselevel='final', serial=0) INFO: os.name: posix INFO: os.uname(): posix.uname_result(sysname='Darwin', nodename='dbnpc148ftp.ads.autodesk.com', release='21.6.0', version='Darwin Kernel Version 21.6.0: Mon Aug 22 20:17:10 PDT 2022; root:xnu-8020.140.49~2/RELEASE_X86_64', machine='x86_64') INFO: sys.getwindowsversion(): N/A --- check: package_management INFO: has conda-meta: False INFO: $VIRTUAL_ENV: '/Users/meltzep/PyCharmProjects/example_raw_scalars/venv' --- check: installed_packages INFO: installed: tensorboard==2.11.0 INFO: installed: tensorflow==2.11.0 INFO: installed: tensorflow-estimator==2.11.0 INFO: installed: tensorboard-data-server==0.6.1 --- check: tensorboard_python_version INFO: tensorboard.version.VERSION: '2.11.0' --- check: tensorflow_python_version 2022-12-18 16:31:42.440164: I tensorflow/core/platform/cpu_feature_guard.cc:193] This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU instructions in performance-critical operations: AVX2 FMA To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags. INFO: tensorflow.__version__: '2.11.0' INFO: tensorflow.__git_version__: 'v2.11.0-rc2-17-gd5b57ca93e5' --- check: tensorboard_data_server_version INFO: data server binary: '/Users/meltzep/PyCharmProjects/example_raw_scalars/venv/lib/python3.9/site-packages/tensorboard_data_server/bin/server' INFO: data server binary version: b'rustboard 0.6.1' --- check: tensorboard_binary_path INFO: which tensorboard: b'/Users/meltzep/PyCharmProjects/example_raw_scalars/venv/bin/tensorboard\n' --- check: addrinfos socket.has_ipv6 = True socket.AF_UNSPEC = socket.SOCK_STREAM = socket.AI_ADDRCONFIG = socket.AI_PASSIVE = Loopback flags: Loopback infos: [(, , 6, '', ('::1', 0, 0, 0)), (, , 6, '', ('127.0.0.1', 0))] Wildcard flags: Wildcard infos: [(, , 6, '', ('::', 0, 0, 0)), (, , 6, '', ('0.0.0.0', 0))] --- check: readable_fqdn INFO: socket.getfqdn(): 'dbnpc148ftp.ads.autodesk.com' --- check: stat_tensorboardinfo INFO: directory: /var/folders/p3/zs5cp__j0dj28_zy3cn35wt80000gp/T/.tensorboard-info INFO: os.stat(...): os.stat_result(st_mode=16895, st_ino=36980904, st_dev=16777220, st_nlink=2, st_uid=502, st_gid=20, st_size=64, st_atime=1671378671, st_mtime=1671381076, st_ctime=1671381076) INFO: mode: 0o40777 --- check: source_trees_without_genfiles INFO: tensorboard_roots (1): ['/Users/meltzep/PyCharmProjects/example_raw_scalars/venv/lib/python3.9/site-packages']; bad_roots (0): [] --- check: full_pip_freeze INFO: pip freeze --all: absl-py==1.3.0 astunparse==1.6.3 cachetools==5.2.0 certifi==2022.12.7 charset-normalizer==2.1.1 flatbuffers==22.12.6 gast==0.4.0 google-auth==2.15.0 google-auth-oauthlib==0.4.6 google-pasta==0.2.0 grpcio==1.51.1 h5py==3.7.0 idna==3.4 importlib-metadata==5.1.0 keras==2.11.0 libclang==14.0.6 Markdown==3.4.1 MarkupSafe==2.1.1 numpy==1.23.5 oauthlib==3.2.2 opt-einsum==3.3.0 packaging==22.0 pip==21.3.1 protobuf==3.19.6 pyasn1==0.4.8 pyasn1-modules==0.2.8 requests==2.28.1 requests-oauthlib==1.3.1 rsa==4.9 setuptools==60.2.0 six==1.16.0 tensorboard==2.11.0 tensorboard-data-server==0.6.1 # Editable install with no version control (tensorboard-plugin-example-raw-scalars==0.1.0) -e /Users/meltzep/PyCharmProjects/example_raw_scalars tensorboard-plugin-wit==1.8.1 tensorflow==2.11.0 tensorflow-estimator==2.11.0 tensorflow-io-gcs-filesystem==0.28.0 termcolor==2.1.1 typing_extensions==4.4.0 urllib3==1.26.13 Werkzeug==2.2.2 wheel==0.37.1 wrapt==1.14.1 zipp==3.11.0 ``````

Browser: Chrome - Version 106.0.5249.119 (Official Build) (x86_64) on Mac OS Monterey 12.6

Issue description

Cannot view the example plugins in tensorboard.

I have cloned tensorboard repo, copied the example tensorboard plugins and followed the instructions in their readmes.

Example Basic

The plugin writes the summaries but no tab shows up in tensorboard to view them. Running --inspect on the logdir returns:

TensorFlow installation not found - running with reduced feature set.
======================================================================
Processing event files... (this can take a few minutes)
======================================================================

Found event files in:
tensorboard_plugin_example/demo_logs

These tags are in tensorboard_plugin_example/demo_logs:
audio -
histograms -
images -
scalars -
tensor
   guestbook
   more_names
======================================================================

Event statistics for tensorboard_plugin_example/demo_logs:
audio -
graph -
histograms -
images -
scalars -
sessionlog:checkpoint -
sessionlog:start -
sessionlog:stop -
tensor
   first_step           0
   last_step            4
   max_step             4
   min_step             0
   num_steps            4
   outoforder_steps     []
======================================================================

Example Raw Scalars

Data is written to summaries and shows up in the standard SCALARS tab, however there is no EXAMPLE_RAW_SCALARS tab.

[qihach64]

Hi @meltzerpete ,

The tag name of the --inspect result seems to indicate that it only contains guestbook and more_names tag for tensorboard_plugin_example/demo_logs directory while in the snapshot you only see the custom_tag tag in tensorboard_plugin_example_raw_scalars/demo_logs. Note that the these are two different directories.

https://github.com/tensorflow/tensorboard#my-tensorboard-isnt-showing-any-data-whats-wrong you can follow the instructions here to verify that you're reading from the correct directory.

If you still have trouble, I'm happy to reproduce the issue first and investigate further.

I have cloned tensorboard repo, copied the example tensorboard plugins and followed the instructions in their readmes.

Could you provide the command you ran and the link to the readme/instructions that you followed?

[meltzerpete]

Hi @qihach64, thanks for the quick response.

Yes, sorry I was not clear - I have tried both example plugins in separate directories with separate environments.

In the first case I followed these steps: https://github.com/tensorflow/tensorboard/tree/master/tensorboard/examples/plugins/example_basic This is where I ran the environment diagnostics script and then the --inspect with the same --logdir directory I was trying to view the plugin in tensorboard.

Next, I started from scratch in a new directory and followed these instructions: https://github.com/tensorflow/tensorboard/tree/master/tensorboard/examples/plugins/example_raw_scalars This is where I took the screenshot from. Of course I do not expect to find the guestbook stuff in this screenshot.

Today, I just tried running tensorboard in each of the directories again, and now it appears to work. I have not changed anything - I know I am using the same --logdir as before for each case. I also notice that in the case of Example Basic if you look at the output from the --inspect you see it says TensorFlow installation not found - running with reduced feature set., however in the environment diagnostic run in the same environment immediately before running --inspect you can see tensorflow is installed INFO: installed: tensorflow==2.11.0. I no longer get this not found message when running --inspect.

The only thing that has changed since I first tried to get these examples, is that I'm in a new shell session. Could this be what caused the issue? I even cleared my browser cache when trying to view these initially to ensure my browser was not causing the issue.

[meltzerpete]

It also seems that uninstalling tensorflow stops tensorboard from finding the plugins, even though tensorboard is still installed.

[qihach64]

Hi @meltzerpete ,

Thanks for providing the steps to reproduce the issue! I tried both plugin examples and they worked just fine for me.

Based on your description of the issue, it seems like that your shell environment is somehow inconsistent across different shell sessions.

Have you been following https://github.com/tensorflow/tensorboard/blob/master/DEVELOPMENT.md#how-to-develop-tensorboard to set up a clean virtualenv? Using virtualenv to develop Tensorboard provides a clean shell environment with the required dependencies.

[meltzerpete]

Thanks.

Yes I am using virtualenv which can be seen in the environment diagnostics.

The issue that is enabling/disabling the plugins for appearing for me is whether or not tensorflow is installed into the venv. If I do a pip uninstall tensorflow they disappear and a pip install tensorflow brings them back.

Are tensorboard plugins supposed to only work with tensorflow installed? Since I am trying to write a plugin that can also be used with pytorch projects, it seems overkill to expect any users to also install tensorflow. Surely tensorboard installation should be enough for a tensorboard plugin? Or is there something wrong here?

[qihach64]

Are tensorboard plugins supposed to only work with tensorflow installed?

Sorry for misunderstanding your questions/concerns. Yes, unfortunately that's the case.

According to https://github.com/tensorflow/tensorboard/blob/master/README.md#can-i-run-tensorboard-without-a-tensorflow-installation :

TensorBoard 1.14+ can be run with a reduced feature set if you do not have TensorFlow installed. The primary limitation is that as of 1.14, only the following plugins are supported: scalars, custom scalars, image, audio, graph, projector (partial), distributions, histograms, text, PR curves, mesh.

As for

it seems overkill to expect any users to also install tensorflow. Surely tensorboard installation should be enough for a tensorboard plugin? Or is there something wrong here?

AFAIK, this issue is under our radar but we don't have a clear timeline to resolve it yet due to some complexity. I will keep this issue open so that we could prioritize this if more and more users are raising similar concerns.

[meltzerpete]

Ah I see. Sorry I had missed that note in the readme.

Thanks for your time, it's saved me a lot of head scratching. I'll watch this space for future developments on this.

[nfelt]

So, the note in the README about plugin support without TensorFlow is actually just about "stock" plugins that are bundled with TensorBoard. For third-party plugins, they should be loadable even if you don't have TF installed as long as the plugins themselves don't require TF.

The issue that is enabling/disabling the plugins for appearing for me is whether or not tensorflow is installed into the venv. If I do a pip uninstall tensorflow they disappear and a pip install tensorflow brings them back.

I'm not sure why this would be happening, but it might be environment-specific. If possible, can you provide repro instructions (e.g. the exact commands you run starting with a totally fresh virtualenv)? Or if not, at least providing the logs from TensorBoard when you start it up and don't see the expected example plugins - you can print logs by adding --verbosity=2.

[meltzerpete]

Sure. Starting from having cloned the tensorboard repo, I do the following:

cp -r tensorboard/tensorboard/examples/plugins/example_basic example_basic
cd example_basic
python -m venv venv    # python -V gives 3.9.5
source venv/bin/activate

pip install tensorboard
python setup.py develop

tensorboard --logdir ./

Now at this stage, although there is no data written, I would expect to see the PLUGIN_BASIC in the dropdown list of tabs where all other plugins are shown. I do not see it there.

Now I continue:

pip install tensorflow
tensorboard --logdir ./
> TensorFlow installation not found - running with reduced feature set.
>
> NOTE: Using experimental fast data loading logic. To disable, pass
>     "--load_fast=false" and report issues on GitHub. More details:
>     https://github.com/tensorflow/tensorboard/issues/4784
>
> Serving TensorBoard on localhost; to expose to the network, use a proxy or pass --bind_all
> TensorBoard 2.6.0 at http://localhost:6006/ (Press CTRL+C to quit)

Note: tensorflow installation is not found.

Now I restart my terminal and navigate back to the example_basic directory.

source venv/bin/activate
tensorboard --logdir ./
>  This TensorFlow binary is optimized with oneAPI Deep Neural Network Library (oneDNN) to use the following CPU 
> instructions in performance-critical operations:  AVX2 FMA
> To enable them in other operations, rebuild TensorFlow with the appropriate compiler flags.
> 
> NOTE: Using experimental fast data loading logic. To disable, pass
>     "--load_fast=false" and report issues on GitHub. More details:
>     https://github.com/tensorflow/tensorboard/issues/4784
> 
> Serving TensorBoard on localhost; to expose to the network, use a proxy or pass --bind_all
> TensorBoard 2.11.0 at http://localhost:6006/ (Press CTRL+C to quit)

Note tensorflow installation is now found. And when I look in tensorboard at the plugin dropdown, I see EXAMPLE_BASIC listed as expected.

Now if I uninstall tensorflow and try again:

pip uninstall tensorflow
tensorboard --logdir ./

The plugin is now gone.

If I run tensorboard now with verbosity=2, this is what I get: https://gist.github.com/meltzerpete/a28facfd2b62cb84be611af4d4fb6b4f (sorry too long to fit in here).

If I restart my shell again, I still cannot see the plugin. But re-installing tensorflow brings is back.

As far as I can tell by looking at the plugin code, there are no explicit dependencies on tensorflow.