tensorflow / tensorboard

TensorFlow's Visualization Toolkit
Apache License 2.0
6.65k stars 1.65k forks source link

add_images will deteriorate image quality #6209

Open Freed-Wu opened 1 year ago

Freed-Wu commented 1 year ago

Issue description

import math
import numpy as np
from matplotlib import pyplot as plt
from torch import Tensor
from torchvision.transforms.functional import pad

from compressai.zoo import image_models
from torch.utils.tensorboard.writer import SummaryWriter
from torchvision.transforms import ToTensor
from PIL import Image

def patch_images(
    samples: Tensor, ratio: int = 1, S: int = 64
) -> tuple[Tensor, list[int]]:
    h, w = samples.shape[-2:]
    h_pad = S * math.ceil(h / S) - h
    w_pad = S * math.ceil(w / S) - w
    l_pad, u_pad = w_pad // 2, h_pad // 2
    r_pad, d_pad = w_pad - l_pad, h_pad - u_pad
    pad_param = [l_pad, u_pad, r_pad, d_pad]
    samples = pad(samples, pad_param)
    crop_param = [ratio * u_pad, ratio * l_pad, ratio * h, ratio * w]
    return samples, crop_param

def convert_images(inputs) -> np.ndarray:
    if isinstance(inputs, np.ndarray):
        img = inputs
    else:
        from torchvision.utils import make_grid

        img = (
            make_grid(inputs.clip(0, 1))
            .permute(1, 2, 0)
            .detach()
            .cpu()
            .float()
            .numpy()
        )
    return img

def plot(inputs: "Tensor | np.ndarray") -> None:
    r"""Plot tensors or array. Clip to $[0, 1]$ by default.

    :param inputs:
    :type inputs: Tensor | np.ndarray
    :rtype: None
    """
    img = convert_images(inputs)
    plt.imshow(img)
    plt.show()

model = image_models["mbt2018"](8, pretrained=True)
img = Image.open("a.png")
images = ToTensor()(img).unsqueeze(0)[:, :3]
padded, params = patch_images(images)
outputs = model(padded)["x_hat"]
plot(outputs)

with SummaryWriter("/output/logs") as writer:
    writer.add_images("a", outputs)

In matplotlib, a.png is displayed normally:

screen-2023-02-25-00-18-06

In tensorbaord, a.png is displayed weirdly:

screen-2023-02-25-00-18-50

a.png is here:

a.png.zip

Environment information (required)

Please run diagnose_tensorboard.py (link below) in the same environment from which you normally run TensorFlow/TensorBoard, and paste the output here:

Diagnostics

Diagnostics output `````` --- check: autoidentify INFO: diagnose_tensorboard.py version 516a2f9433ba4f9c3a4fdb0f89735870eda054a1 --- check: general INFO: sys.version_info: sys.version_info(major=3, minor=10, micro=9, releaselevel='final', serial=0) INFO: os.name: posix INFO: os.uname(): posix.uname_result(sysname='Linux', nodename='desktop', release='6.1.12-arch1-1', version='#1 SMP PREEMPT_DYNAMIC Tue, 14 Feb 2023 22:08:08 +0000', machine='x86_64') INFO: sys.getwindowsversion(): N/A --- check: package_management INFO: has conda-meta: False INFO: $VIRTUAL_ENV: None --- check: installed_packages INFO: installed: tensorboard==2.11.0 WARNING: no installation among: ['tensorflow', 'tensorflow-gpu', 'tf-nightly', 'tf-nightly-2.0-preview', 'tf-nightly-gpu', 'tf-nightly-gpu-2.0-preview'] WARNING: no installation among: ['tensorflow-estimator', 'tensorflow-estimator-2.0-preview', 'tf-estimator-nightly'] INFO: installed: tensorboard-data-server==0.7.0a0 --- check: tensorboard_python_version INFO: tensorboard.version.VERSION: '2.11.0' --- check: tensorflow_python_version Traceback (most recent call last): File "/dev/shm/./diagnose_tensorboard.py", line 528, in main suggestions.extend(check()) File "/dev/shm/./diagnose_tensorboard.py", line 81, in wrapper result = fn() File "/dev/shm/./diagnose_tensorboard.py", line 284, in tensorflow_python_version import tensorflow as tf ModuleNotFoundError: No module named 'tensorflow' --- check: tensorboard_data_server_version INFO: data server binary: '/usr/lib/python3.10/site-packages/tensorboard_data_server/bin/server' INFO: data server binary version: b'rustboard 0.7.0-alpha.0' --- check: tensorboard_binary_path INFO: which tensorboard: b'/usr/bin/tensorboard\n' --- check: addrinfos socket.has_ipv6 = True socket.AF_UNSPEC = socket.SOCK_STREAM = socket.AI_ADDRCONFIG = socket.AI_PASSIVE = Loopback flags: Loopback infos: [(, , 6, '', ('::1', 0, 0, 0)), (, , 6, '', ('127.0.0.1', 0))] Wildcard flags: Wildcard infos: [(, , 6, '', ('0.0.0.0', 0)), (, , 6, '', ('::', 0, 0, 0))] --- check: readable_fqdn INFO: socket.getfqdn(): 'localhost' --- check: stat_tensorboardinfo INFO: directory: /tmp/.tensorboard-info INFO: os.stat(...): os.stat_result(st_mode=16895, st_ino=11056, st_dev=32, st_nlink=2, st_uid=1000, st_gid=1000, st_size=60, st_atime=1677246960, st_mtime=1677253463, st_ctime=1677253463) INFO: mode: 0o40777 --- check: source_trees_without_genfiles INFO: tensorboard_roots (1): ['/usr/lib/python3.10/site-packages']; bad_roots (0): [] --- check: full_pip_freeze INFO: pip freeze --all: absl-py==1.3.0 agate==1.6.3 agate-dbf==0.2.2 agate-excel==0.2.5 agate-sql==0.5.8 aiohttp==3.8.3 aiosignal==1.3.1 alabaster==0.7.13 albumentations==1.3.0 ansi2html==1.8.0 ansicolor==0.3.2 anyio==3.6.2 appdirs==1.4.4 argon2-cffi==21.3.0 argon2-cffi-bindings==21.2.0 argopt==0.7.1 aria2p==0.11.3 arrow==1.2.3 asciimatics==1.14.0 asciinema==2.2.0 asgiref==3.5.2 asttokens==2.2.1 async-generator==1.10 async-timeout==4.0.2 attrs==22.2.0 autocommand==2.2.2 Babel==2.11.0 backcall==0.2.0 beautifulsoup4==4.11.2 black==23.1.0 bleach==6.0.0 blessed==1.20.0 boolean.py==4.0 Brotli==1.0.9 btrfsutil==6.1.3 build==0.10.0 CacheControl==0.12.11 cachetools==5.3.0 cachy==0.3.0 cattrs==22.2.0 Cerberus==1.3.4 certifi==2022.12.7 cffi==1.15.1 cfgv==3.3.1 chardet==5.1.0 charset-normalizer==2.1.1 cleo==2.0.1 click==8.1.3 cmake-annotate==0.6.13 cmake-format==0.6.13 cmake-lint==0.6.13 cmake-parse==0.6.13 cmakelang==0.6.13 cmakelint==1.4.2 codecov==2.1.12 colorama==0.4.6 comm==0.1.2 compressai==1.2.1 contourpy==1.0.7 coverage==6.5.0 crashtest==0.4.1 cryptography==39.0.1 cssselect==1.2.0 csvkit==1.0.7 cupshelpers==1.0 cupy==11.5.0 cycler==0.11.0 Cython==0.29.33 datasets==1.17.0 dbfread==2.0.7 debugpy==1.6.6 decorator==5.1.1 defusedxml==0.7.1 deprecation==2.1.0 dill==0.3.6 distlib==0.3.6 Django==4.1.7 dnspython==2.2.1 docker-pycreds==0.4.0 docopt==0.6.2 docopts==0.6.1+fix docstring-to-markdown==0.11 docutils==0.19 dominate==2.7.0 doq==0.9.1 dulwich==0.21.3 dunamai==1.15.0 einops==0.6.0 elementpath==3.0.2 emoji-fzf==0.7.0 entrypoints==0.4 et-xmlfile==1.1.0 eventlet==0.33.2 exceptiongroup==1.1.0 execnet==1.9.0 executing==1.2.0 fastjsonschema==2.16.2 fastrlock==0.8.1 filelock==3.8.2 flexmock==0.11.3 fonttools==4.38.0 frozenlist==1.3.3 fsspec==2022.11.0 future==0.18.2 fuzzywuzzy==0.18.0 gdown==4.6.3 gevent==22.10.2 gitdb==4.0.9 gitlint-core==0.17.0 GitPython==3.1.29 gmpy2==2.1.5 google-auth==2.14.1 google-auth-oauthlib==0.7.1 gpustat==1.0.0 greenlet==2.0.2 grpcio==1.51.1 gunicorn==20.1.0 h5py==3.7.0 help2man==0.0.1 html5lib==1.1 huggingface-hub==0.12.1 HyFetch==1.4.7 hypothesis==6.59.0 identify==2.5.18 idna==3.4 imageio==2.25.0 imagesize==1.4.1 imgaug==0.4.0 importlib-metadata==5.0.0 inflect==6.0.2 iniconfig==2.0.0 installer==0.6.0 iopath==0.1.9 ipykernel==6.21.2 ipython==8.10.0 ipython-genutils==0.2.0 isodate==0.6.1 isort==5.12.0 jaraco.classes==3.2.3 jaraco.context==4.3.0 jaraco.functools==3.5.2 jaraco.text==3.11.1 jarowinkler==1.2.3 jedi==0.18.2 jedi-language-server==0.40.0 jeepney==0.8.0 Jinja2==3.1.2 joblib==1.2.0 -e git+https://github.com/Freed-Wu/jpegai@d7d0c0e8e00f51c15807047b5343d7e8549247e2#egg=jpegai json5==0.9.12 jsonpatch==1.32 jsonpointer==2.3 jsonschema==4.17.3 jupyter-events==0.6.3 jupyter_client==8.0.3 jupyter_core==5.2.0 jupyter_packaging==0.12.3 jupyter_server==2.3.0 jupyterlab==3.6.1 jupyterlab-pygments==0.2.2 jupyterlab_server==2.19.0 kaggle==1.5.12 keyring==23.11.0 keyring-pass==0.7.1 kiwisolver==1.4.4 langdetect==1.0.9 lark==1.1.5 latexify-py==0.2.0 lazy_loader==0.1 leather==0.3.4 lensfun==0.3.3 Levenshtein==0.20.9 license-expression==30.1.1.dev0+g1548127.d20230117 lit==15.0.7.dev0 lmdb==1.4.0 lockfile==0.12.2 loguru==0.6.0 lpips==0.1.4 lsprotocol==2022.0.0a10 lxml==4.9.2 Markdown==3.4.1 markdown-it-py==2.1.0 MarkupSafe==2.1.2 matplotlib==3.6.2 matplotlib-inline==0.1.6 mdformat==0.7.16 mdit-py-plugins==0.3.1 mdurl==0.1.2 mistune==2.0.5 mock==3.0.5 monotonic==1.6 more-itertools==9.0.0 mpmath==1.2.1 msgpack==1.0.4 multidict==6.0.3 multiprocess==0.70.14 munch==2.5.0 mypy-extensions==1.0.0 myst-parser==0.18.1 namcap==3.3.1 nbclient==0.7.2 nbconvert==7.2.9 nbformat==5.7.3 nest-asyncio==1.5.6 netease-musicbox==0.3.1 netsnmp-python==1.0a1 networkx==2.8.8 nodeenv==1.7.0 nose==1.3.7 notebook_shim==0.2.2 numpy==1.24.2 nvidia-ml-py==11.525.84 oauthlib==3.2.2 olefile==0.46 OpenCC==1.1.5 opencv-python==4.7.0.68 openpyxl==3.1.0 opt-einsum==3.3.0 ordered-set==4.1.0 packaging==23.0 pandas==1.5.3 pandocfilters==1.5.0 parsedatetime==2.6 parso==0.8.3 pathspec==0.11.0 pathtools==0.1.2 pbr==5.11.1 pdm-pep517==1.1.2 pexpect==4.8.0 pickleshare==0.7.5 Pillow==9.4.0 pip==23.0.1 pip-cache==0.0.6 pip-tools==6.12.2 pix2tex==0.0.29 pkginfo==1.9.6 platformdirs==2.6.2 plotext==5.2.8 pluggy==1.0.0 ply==3.11 poetry==1.3.2 poetry-core==1.5.0 poetry-plugin-export==1.2.0 pooch==1.6.0 portalocker==2.7.0 powerline-status==2.8.3 pre-commit==2.20.0 prometheus-client==0.15.0 promise==2.3 prompt-toolkit==3.0.36 protobuf==4.21.12 psutil==5.9.4 ptpython==3.0.21 ptyprocess==0.7.0 pudb==2022.1.3 pure-eval==0.2.2 py-notifier==0.3.2 pyalpm==0.10.6 pyaml==21.10.1 pyarrow==10.0.1 pyasn1==0.4.8 pyasn1-modules==0.2.8 pybind11==2.10.3 pycairo==1.23.0 pycparser==2.21 pycryptodomex==3.12.0 pycups==2.0.1 pycurl==7.45.2 pydantic==1.10.4 pydocstyle==6.1.1 pyelftools==0.29 pyfiglet==0.8.post1 pygls==1.0.1 Pygments==2.14.0 PyGObject==3.42.2 PyICU==2.10.2 pynvim==0.4.3 pynvml==11.4.1 PyOpenGL==3.1.6 pyOpenSSL==23.0.0 pyparsing==3.0.9 pyperclip==1.8.2 pyproject_hooks==1.0.0 PyQt3D==5.15.6 PyQt5==5.15.9 PyQt5-sip==12.11.1 PyQtChart==5.15.6 PyQtDataVisualization==5.15.5 PyQtNetworkAuth==5.15.5 PyQtPurchasing==5.15.5 PyQtWebEngine==5.15.6 pyrsistent==0.19.3 PySocks==1.7.1 PyStarDict @ file:///home/wzy/.cache/yay/python-pystardict/src/PyStarDict-0.8-py2.py3-none-any.whl pytest==7.2.1 pytest-cov==4.0.0 pytest-pudb==0.7.0 pytest-xdist==3.2.0 python-dateutil==2.8.2 python-json-logger==2.0.5 python-slugify==8.0.0 python-wakatime==0.0.3 pytimeparse==1.1.8 pytorch-msssim==0.2.1 pytz==2022.7 PyWavelets==1.4.1 pyxdg==0.28 PyYAML==6.0 pyzmq==24.0.1 qudida==0.0.4 rapidfuzz==2.13.7 readme-renderer==37.3 regex==2022.10.31 repl-python-codestats==0.0.3 repl-python-wakatime==0.0.5 requests==2.28.1 requests-cache==0.9.6 requests-oauthlib==1.3.1 requests-toolbelt==0.10.1 requests-unixsocket==0.3.0 rfc3986==2.0.0 rich==13.3.1 rsa==4.9 sacremoses==0.0.35 SciencePlots==2.0.1 scikit-image==0.20.0rc6 scikit-learn==1.2.0 scipy==1.10.1 sebaubuntu-libs==1.3.1 SecretStorage==3.3.3 Send2Trash==1.8.0 sentry-sdk==1.11.0 setproctitle==1.3.2 setuptools==67.3.0 setuptools-scm==7.1.0 sh==1.14.3 Shapely==1.8.5.post1 shellingham==1.5.0.post1 shortuuid==1.0.11 shtab==1.5.8 six==1.16.0 smmap==5.0.0 sniffio==1.3.0 snowballstemmer==2.2.0 sortedcontainers==2.4.0 soupsieve==2.4 Sphinx==5.3.0 sphinx-rtd-theme==1.1.1 sphinxcontrib-applehelp==1.0.4 sphinxcontrib-devhelp==1.0.2 sphinxcontrib-eval==0.0.3 sphinxcontrib-htmlhelp==2.0.1 sphinxcontrib-jsmath==1.0.1 sphinxcontrib-qthelp==1.0.3 sphinxcontrib-serializinghtml==1.1.5 SQLAlchemy==1.4.44 sqlparse==0.4.3 stack-data==0.6.2 super-image @ https://test-files.pythonhosted.org/packages/42/c4/8b9cca01f45246017791440e70cef1ca52614daae317faaa49295d233c24/super_image-0.1.7-py3-none-any.whl sympy==1.11.1 tabulate==0.9.0 TBB==0.2 team==1.0 tensorboard==2.11.0 tensorboard-data-server==0.7.0a0 tensorboard-plugin-wit==1.8.1 tensorboardX==2.5.1 termcolor==2.2.0 terminado==0.17.1 testpath==0.6.0 text-unidecode==1.3 threadpoolctl==3.1.0 tifffile==2023.2.3 timm==0.6.12 tinycss2==1.2.1 tokenizers==0.13.2 toml==0.10.2 tomli==2.0.1 tomli_w==1.0.0 tomlkit==0.11.6 torch==1.13.1 torch-fidelity==0.3.0 -e git+https://github.com/Harry24k/adversarial-attacks-pytorch@320d15dd7b2234b25a389f0fee6ef23b02c46fb7#egg=torchattacks torchdata==0.5.1 torchfile==0.1.0 torchinfo==1.7.1 torchmetrics==0.11.1 torchvision==0.14.1a0 tornado==6.2 tqdm==4.64.1 traitlets==5.9.0 transformers==4.26.1 translate-shell==0.0.11 trash-cli==0.22.10.20 trove-classifiers==2023.2.9 twine==4.0.2 typeguard==2.13.3 typing_extensions==4.5.0 uc-micro-py==1.0.1 url-normalize==1.4.3 urllib3==1.26.12 urwid==2.1.2 urwid-readline==0.13 uvloop==0.17.0 validate-pyproject==0.12.1 validators==0.20.0 vim-vint==0.4a4.dev14+ge120918 vimdoc==0.6.0 virtualenv==20.17.1 visdom==0.2.4 visidata==2.11 wandb==0.13.5 wcwidth==0.2.5 webencodings==0.5.1 websocket-client==1.5.1 Werkzeug==2.2.3 wheel==0.38.4 wordcloud==1.8.1 wxPython==4.2.0 x-transformers==1.2.3 xlrd==2.0.1 xmlschema==2.2.1 xxhash==3.2.0 yamllint==1.29.0 yapf==0.32.0 yarl==1.8.2 you-get==0.4.1650 youtube-dl==2021.12.17 zipp==3.14.0 zope.event==4.6 zope.interface==5.5.2 ``````

Next steps

No action items identified. Please copy ALL of the above output, including the lines containing only backticks, into your GitHub issue or comment. Be sure to redact any sensitive information.

For browser-related issues, please additionally specify:

rileyajones commented 1 year ago

I'm not familiar with the torchvision library, are you sure the images it generates adhere to our documented compatible images? https://www.tensorflow.org/api_docs/python/tf/summary/image