tensorflow / tensorboard

TensorFlow's Visualization Toolkit
Apache License 2.0
6.68k stars 1.65k forks source link

Large graphs fail to render correctly #2707

Open Bidski opened 4 years ago

Bidski commented 4 years ago

Environment information (required)

Please run diagnose_tensorboard.py (link below) in the same environment from which you normally run TensorFlow/TensorBoard, and paste the output here:

Diagnostics

Diagnostics output `````` --- check: autoidentify INFO: diagnose_tensorboard.py version 4725c70c7ed724e2d1b9ba5618d7c30b957ee8a4 --- check: general INFO: sys.version_info: sys.version_info(major=3, minor=7, micro=4, releaselevel='final', serial=0) INFO: os.name: posix INFO: os.uname(): posix.uname_result(sysname='Linux', nodename='bidski-laptop', release='5.3.1-arch1-1-ARCH', version='#1 SMP PREEMPT Sat Sep 21 11:33:49 UTC 2019', machine='x86_64') INFO: sys.getwindowsversion(): N/A --- check: package_management INFO: has conda-meta: False INFO: $VIRTUAL_ENV: None --- check: installed_packages INFO: installed: tb-nightly==2.0.0a20190917 INFO: installed: tf-nightly==1.14.0 INFO: installed: tf-estimator-nightly==1.14.0.dev2019092001 --- check: tensorboard_python_version INFO: tensorboard.version.VERSION: '2.0.0a20190917' --- check: tensorflow_python_version 2019-10-01 10:11:24.039418: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1 INFO: tensorflow.__version__: '1.14.0' INFO: tensorflow.__git_version__: '1.15.0-rc1-256-g5dfacce7a7' --- check: tensorboard_binary_path INFO: which tensorboard: b'/usr/bin/tensorboard\n' --- check: readable_fqdn INFO: socket.getfqdn(): 'bidski-laptop.localdomain' --- check: stat_tensorboardinfo INFO: directory: /tmp/.tensorboard-info INFO: os.stat(...): os.stat_result(st_mode=16895, st_ino=124761, st_dev=47, st_nlink=2, st_uid=1000, st_gid=1000, st_size=40, st_atime=1569885090, st_mtime=1569887967, st_ctime=1569887967) INFO: mode: 0o40777 --- check: source_trees_without_genfiles INFO: tensorboard_roots (1): ['/usr/lib/python3.7/site-packages']; bad_roots (0): [] --- check: full_pip_freeze INFO: pip freeze --all: absl-py==0.8.0 alabaster==0.7.12 anytree==2.6.0 apipkg==1.5 appdirs==1.4.3 asn1crypto==0.24.0 astor==0.8.0 astroid==2.2.5 atomicwrites==1.3.0 attrs==19.1.0 awscli==1.16.242 Babel==2.7.0 backcall==0.1.0 bcrypt==3.1.7 black==19.3b0 boto==2.49.0 botocore==1.12.232 Brlapi==0.7.0 btrfsutil==1.1.1 CacheControl==0.12.5 cached-property==1.5.1 cffi==1.12.3 chardet==3.0.4 chrome-gnome-shell==0.0.0 Click==7.0 cmake-format==0.5.5 colorama==0.3.9 coverage==4.5.4 cryptography==2.7 cycler==0.10.0 decorator==4.4.0 distlib==0.2.9 distro==1.4.0 docker==4.0.2 docker-compose==1.24.1 docker-pycreds==0.4.0 dockerpty==0.4.1 docopt==0.6.2 docutils==0.15.2 entrypoints==0.3 execnet==1.7.1 flake8==3.7.8 gast==0.2.2 google-pasta==0.1.7 greenlet==0.4.15 grpcio==1.23.0 h5py==2.10.0 html5lib==1.0.1 idna==2.8 imagesize==1.1.0 importlib-metadata==0.23 iniconfig==1.0.0 ipython==7.6.1 ipython-genutils==0.1.0 isc==2.0 isort==4.3.21 jedi==0.15.1 Jinja2==2.10.1 jmespath==0.9.4 jsonschema==3.0.2 Keras-Applications==1.0.8 Keras-Preprocessing==1.0.5 kiwisolver==1.1.0 lazy-object-proxy==1.4.2 lensfun==0.3.2 lit==0.8.1.dev0 lockfile==0.12.2 louis==3.11.0 lxml==4.4.1 Markdown==3.1.1 MarkupSafe==1.1.1 matplotlib==3.1.1 mccabe==0.6.1 meson==0.51.2 mock==3.0.5 more-itertools==4.3.0 msgpack==0.6.1 mypy==0.720 mypy-extensions==0.4.1 namcap==3.2.7 networkx==2.3 numpy==1.17.2 opt-einsum==3.0.1 packaging==19.1 pandas==0.25.1 paramiko==2.6.0 parso==0.5.1 pbr==5.4.3 pep517==0.6.0 pexpect==4.7.0 pickleshare==0.7.5 Pillow==6.1.0 pip==19.0.3 Pivy==0.6.5a1 pluggy==0.12.0 ply==3.11 progress==1.5 prompt-toolkit==2.0.9 protobuf==3.7.0 psutil==5.6.3 ptyprocess==0.6.0 pwquality==1.4.1 py==1.8.0 pyalpm==0.8.5 pyasn1==0.4.7 pycairo==1.18.1 pycodestyle==2.5.0 pycparser==2.19 pyelftools==0.25 pyflakes==2.1.1 Pygments==2.4.2 PyGObject==3.34.0 pygraphviz==1.5 pylint==2.3.1 PyNaCl==1.3.0 pynvim==0.3.2 pyparsing==2.4.2 pyrsistent==0.15.4 pytest==5.1.3 pytest-black==0.3.7 pytest-cov==2.7.1 pytest-flake8==1.0.4 pytest-forked==1.0.2 pytest-mypy==0.4.1 pytest-pylint==0.14.1 pytest-xdist==1.29.0 python-dateutil==2.8.0 pytoml==0.1.21 pytz==2019.2 pyxdg==0.26 PyYAML==5.1.2 requests==2.22.0 retrying==1.3.3 rope==0.14.0 ropemode==0.5 ropevim==0.7.0 rsa==3.4.2 s3transfer==0.2.1 scipy==1.3.1 setuptools==41.2.0 simplegeneric==0.8.1 simplejson==3.16.1 six==1.12.0 snowballstemmer==1.2.1 Sphinx==2.2.0 sphinxcontrib-applehelp==1.0.1 sphinxcontrib-devhelp==1.0.1 sphinxcontrib-htmlhelp==1.0.2 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.2 sphinxcontrib-serializinghtml==1.1.3 tb-nightly==2.0.0a20190917 team==1.0 termcolor==1.1.0 texttable==1.6.2 tf-estimator-nightly==1.14.0.dev2019092001 tf-nightly==1.14.0 toml==0.10.0 traitlets==4.3.2 typed-ast==1.4.0 typing-extensions==3.7.4 urllib3==1.25.5 wcwidth==0.1.7 webencodings==0.5.1 websocket-client==0.56.0 Werkzeug==0.16.0 wheel==0.33.6 wrapt==1.11.2 xdot==1.1 zipp==0.6.0 ``````

Next steps

No action items identified. Please copy ALL of the above output, including the lines containing only backticks, into your GitHub issue or comment. Be sure to redact any sensitive information.

For browser-related issues, please additionally specify:

Issue description

Please describe the bug as clearly as possible. How can we reproduce the problem without additional resources (including external data files and proprietary Python modules)?

When loading a large graph the graph view will take a very long time to load. It is not uncommon for chrome to report the page as unresponsive during loading. Eventually the graph will display, for a couple of seconds, however the screen will then become blank as is shown in the screenshot. There are no errors displayed in the chrome developer console, nor in the console where tensorboard was started.

Attached is an event file that produces this issue. It does not appear to be a RAM issue (run on a 32-core system with 128GB RAM and 4 GTX1080Tis and no significant resource usage was observed).

logs.tar.gz

nfelt commented 4 years ago

Thanks for the clear description and the sample file, I can reproduce the issue - I saw a rendered graph briefly but then it blanked out, and it appears that the visualization is just choking on the size.

We're working on improvements to the graph visualization to address some of the current scaling limits, please stay tuned.

cc @davidsoergel FYI