`huggingface_hub.notebook_login` fails on AWS Sagemaker Studio

[david-waterworth]

Describe the bug

It looks like there's a compatibility issue between the version of jupyter used by AWS Sagemaker Studio, ipywidgets and/or huggingface_hub

I've installed the latest versions of transformers and datasets and ipywidgets and the output of notebook_login wont render.

Also note in the System info - Running in notebook ?: No - but I am running in a notebook, it seems the notebook detection code doesn't recognise the AWS Sagemaker Studio notebook variant.

As a workaround I'm using the cell magic

!huggingface-cli login --token $TOKEN

Reproduction

from huggingface_hub import notebook_login
notebook_login()

Error displaying widget: model not found

Logs

No response

System info

- huggingface_hub version: 0.16.4
- Platform: Linux-4.14.314-238.539.amzn2.x86_64-x86_64-with-glibc2.31
- Python version: 3.10.8
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Token path ?: /root/.cache/huggingface/token
- Has saved token ?: True
- Who am I ?: david-waterworth
- Configured git credential helpers: 
- FastAI: N/A
- Tensorflow: N/A
- Torch: 2.0.1
- Jinja2: 3.1.2
- Graphviz: N/A
- Pydot: N/A
- Pillow: N/A
- hf_transfer: N/A
- gradio: N/A
- tensorboard: N/A
- numpy: 1.25.1
- pydantic: N/A
- aiohttp: 3.8.5
- ENDPOINT: https://huggingface.co
- HUGGINGFACE_HUB_CACHE: /root/.cache/huggingface/hub
- HUGGINGFACE_ASSETS_CACHE: /root/.cache/huggingface/assets
- HF_TOKEN_PATH: /root/.cache/huggingface/token
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False

[david-waterworth]

Also note the Error displaying widget: model not found occurs frequently, i.e. in place of the download progress bars when downloading model weights, datasets etc.

[Wauplin]

Hi @david-waterworth , I'm sorry you're experiencing this issue. As you said, there is most likely a compatibility issue between the installed packages. I tried it myself by just installing huggingface_hub via pip in a AWS Sagemaker Studio session and it worked just fine.

Could you run conda list | grep "jupyter\|widget" and check if maybe you have an missing/outdated package somewhere that breaks the widgets? Here is a screenshot of what I am getting on my side:

[Wauplin]

Also note in the System info - Running in notebook ?: No - but I am running in a notebook, it seems the notebook detection code doesn't recognise the AWS Sagemaker Studio notebook variant.

This is because you are running !huggingface-cli env in a notebook which under the hood call the actual command line in a terminal. To get the information from a notebook, you should use

from huggingface_hub import dump_environment_info

dump_environment_info()

I admit that this is not really well documented but also not very problematic :)

[david-waterworth]

@Wauplin I'm guessing from your suggestion that you used the Data Science kernel. I switched to that and confirmed it does work fine.

But it doesn't work on the Base Python 3.0 kernel (which isn't based on conda and ships with py310 rather than py37 in Data Science kernel).

The output of the equivalent %pip list | grep "widget\|jupyter on a fresh instance is:

jupyter_client     8.2.0
jupyter_core       5.3.1

After %pip install ipywidgets

ipywidgets         8.0.7
jupyter_client     8.2.0
jupyter_core       5.3.1
jupyterlab-widgets 3.0.8
widgetsnbextension 4.0.8

I restarted the kernel then tried to run notebook_login() and it didn't render properly

What did work (with a clean kernel) was:

%pip install ipywidgets==7.7.5

The output of %pip list | grep "widget\|jupyter is then

ipywidgets                7.7.5
jupyter_client            8.2.0
jupyter_core              5.3.1
jupyter-events            0.6.3
jupyter-lsp               2.2.0
jupyter_server            2.7.0
jupyter_server_terminals  0.4.4
jupyterlab                4.0.3
jupyterlab-pygments       0.2.2
jupyterlab_server         2.24.0
jupyterlab-widgets        1.1.4
widgetsnbextension        3.6.4

Also at this stage %pip install -U ipywidget breaks it again - so looks like huggingface_hub requires ipywidget<8.0.0

[Wauplin]

Thanks for investigating this @david-waterworth. I tried locally and for me the notebook login works when I have ipywidgets==8.0.2 installed so I expect the issue to be more tedious (don't know well AWS Sagemaker studio TBH but there has to be a difference compared to a local setup).

In any case, I'm glad that %pip install ipywidgets==7.7.5 fixes your issues. Not the ideal solution but if someone else come across this issue, at least there is a workaround. In the meantime, I'll try to investigate the real issue. I'm reluctant to pin the version of ipywidgets has it could cause problem to other users while being valid in a vast majority of cases (i.e. when not running in AWS sagemaker studio).

[anjanvb]

I can confirm that I am getting the same error on Pytorch 1.13 Python 3.9 and Pytorch 2.0.0 Python 3.10 Kernels on SM Studio. Using ipywidgets==8.0.2 on these kernels has not solved the issue, But using ipywidgets==7.7.5 did work.