tensorflow / tensorboard

TensorFlow's Visualization Toolkit
Apache License 2.0
6.69k stars 1.66k forks source link

Fix indirect dependency on the package six. #6580

Closed arcra closed 1 year ago

arcra commented 1 year ago

On Sep 12, 2023, our nightly release workflow (in GH Actions) started failing, with the following error:

Traceback (most recent call last):
  File "/tmp/tmp.f1F7QYeaX0/virtualenvs/venv-python3//bin/tensorboard", line 5, in <module>
    from tensorboard.main import run_main
  File "/tmp/tmp.f1F7QYeaX0/virtualenvs/venv-python3/lib/python3.9/site-packages/tensorboard/main.py", line 27, in <module>
    from tensorboard import default
  File "/tmp/tmp.f1F7QYeaX0/virtualenvs/venv-python3/lib/python3.9/site-packages/tensorboard/default.py", line 32, in <module>
    from tensorboard.plugins.audio import audio_plugin
  File "/tmp/tmp.f1F7QYeaX0/virtualenvs/venv-python3/lib/python3.9/site-packages/tensorboard/plugins/audio/audio_plugin.py", line 23, in <module>
    from tensorboard import plugin_util
  File "/tmp/tmp.f1F7QYeaX0/virtualenvs/venv-python3/lib/python3.9/site-packages/tensorboard/plugin_util.py", line 20, in <module>
    from tensorboard._vendor.bleach.sanitizer import Cleaner
  File "/tmp/tmp.f1F7QYeaX0/virtualenvs/venv-python3/lib/python3.9/site-packages/tensorboard/_vendor/bleach/__init__.py", line 5, in <module>
    from tensorboard._vendor.bleach.linkifier import (
  File "/tmp/tmp.f1F7QYeaX0/virtualenvs/venv-python3/lib/python3.9/site-packages/tensorboard/_vendor/bleach/linkifier.py", line 4, in <module>
    from tensorboard._vendor import html5lib
  File "/tmp/tmp.f1F7QYeaX0/virtualenvs/venv-python3/lib/python3.9/site-packages/tensorboard/_vendor/html5lib/__init__.py", line 25, in <module>
    from .html5parser import HTMLParser, parse, parseFragment
  File "/tmp/tmp.f1F7QYeaX0/virtualenvs/venv-python3/lib/python3.9/site-packages/tensorboard/_vendor/html5lib/html5parser.py", line 2, in <module>
    from six import with_metaclass, viewkeys
ModuleNotFoundError: No module named 'six'

Apparently, six is a dependency for two of our "vendored" packages, but we didn't have it as an explicit dependency. It's unclear why it was not an issue up until now. My current hypothesis is that this was included by default by either pip or virtualenv, or it was installed because it was a dependency of some other dependency we do have explicitly, and just recently that stopped being true, but nothing in our setup has changed recently, and the package versions used also seem to be the same for the latest successful run, so I haven't been able to validate or understand how this could be the case.

Both the latest successful run and the first failed one seem to have the same package versions for the pip and virtualenv packages, and even the six package seems to be initially installed with the same version, but somehow the "test pip package" step, which runs our test_pip_package.sh script, which in turn creates a virtualenv, started failing with this No module named 'six' error.

Since this is a dependency of our vendored packages [1, 2], perhaps we should include it as another vendored package, but I'm not quite sure of what this entails. At the same time, I don't know if we should make it an explicit dependency of our product in the requirements.txt, so for now, I lean towards adding it to our requirements_dev.txt file.

arcra commented 1 year ago

Ah, I think I found the issue. In the step where we create the virtualenv, we print pip freeze to see what packages are installed. In the latest successful run, the google-auth==2.22.0 version is installed, whereas in the failing one, google-auth==2.23.0 is used.

I installed both of these versions in a clean environment and validated that v2.22.0 had a dependency on the six package, and the newer one does not. So the conclusion is the same, we were relying on an indirect dependency.