details is always None with `text_generation`

[WesleyTheGeolien]

Describe the bug

When passing details=True and stream=True (I haven't tried with other configurations) details appears to always be None at least in the examples I have run. I am using huggingface_hub=0.23.2 on Linux if that helps. I think I was on 0.20 before and I seem to remember this to be working so it looks like there may have been a regression.

Reproduction

Example taken from InferenceClient.text_generation docstring case 4:

from huggingface_hub import InferenceClient
client = InferenceClient()

for details in client.text_generation("The huggingface_hub library is ", max_new_tokens=12, details=True, stream=True):
    print(details)

Logs

TextGenerationStreamOutput(index=1, token=TextGenerationStreamOutputToken(id=28740, logprob=-0.54541016, special=False, text='1'), details=None, generated_text=None, top_tokens=None)
TextGenerationStreamOutput(index=2, token=TextGenerationStreamOutputToken(id=28734, logprob=-0.10461426, special=False, text='0'), details=None, generated_text=None, top_tokens=None)
TextGenerationStreamOutput(index=3, token=TextGenerationStreamOutputToken(id=28734, logprob=-0.015731812, special=False, text='0'), details=None, generated_text=None, top_tokens=None)
TextGenerationStreamOutput(index=4, token=TextGenerationStreamOutputToken(id=28823, logprob=-0.008117676, special=False, text='%'), details=None, generated_text=None, top_tokens=None)
TextGenerationStreamOutput(index=5, token=TextGenerationStreamOutputToken(id=1565, logprob=-0.60253906, special=False, text=' open'), details=None, generated_text=None, top_tokens=None)
TextGenerationStreamOutput(index=6, token=TextGenerationStreamOutputToken(id=28733, logprob=-0.6123047, special=False, text='-'), details=None, generated_text=None, top_tokens=None)
TextGenerationStreamOutput(index=7, token=TextGenerationStreamOutputToken(id=1394, logprob=-0.0039100647, special=False, text='source'), details=None, generated_text=None, top_tokens=None)
TextGenerationStreamOutput(index=8, token=TextGenerationStreamOutputToken(id=304, logprob=-0.5629883, special=False, text=' and'), details=None, generated_text=None, top_tokens=None)
TextGenerationStreamOutput(index=9, token=TextGenerationStreamOutputToken(id=5242, logprob=-1.7705078, special=False, text=' released'), details=None, generated_text=None, top_tokens=None)
TextGenerationStreamOutput(index=10, token=TextGenerationStreamOutputToken(id=916, logprob=-0.036987305, special=False, text=' under'), details=None, generated_text=None, top_tokens=None)
TextGenerationStreamOutput(index=11, token=TextGenerationStreamOutputToken(id=272, logprob=-0.084228516, special=False, text=' the'), details=None, generated_text=None, top_tokens=None)
TextGenerationStreamOutput(index=12, token=TextGenerationStreamOutputToken(id=8958, logprob=-0.4724121, special=False, text=' Apache'), details=None, generated_text='100% open-source and released under the Apache', top_tokens=None)

System info

- huggingface_hub version: 0.23.2
- Platform: Linux-6.8.0-76060800daily20240311-generic-x86_64-with-glibc2.35
- Python version: 3.10.13
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Token path ?: /home/xxx/.cache/huggingface/token
- Has saved token ?: False
- Configured git credential helpers: 
- FastAI: N/A
- Tensorflow: N/A
- Torch: 2.1.2
- Jinja2: 3.1.3
- Graphviz: N/A
- keras: N/A
- Pydot: N/A
- Pillow: 10.2.0
- hf_transfer: N/A
- gradio: 4.21.0
- tensorboard: N/A
- numpy: 1.26.4
- pydantic: 2.7.1
- aiohttp: 3.9.3
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: /home/xxx/.cache/huggingface/hub
- HF_ASSETS_CACHE: /home/xxx/.cache/huggingface/assets
- HF_TOKEN_PATH: /home/xxx/.cache/huggingface/token
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10

[Wauplin]

Thanks for reporting @WesleyTheGeolien! I have been able to reproduce the issue as well. It seems to be a recent breaking change introduced in https://github.com/huggingface/huggingface_hub/pull/2237. I just opened https://github.com/huggingface/huggingface_hub/pull/2316 to fix it and will make a hot-fix release once reviewed and merged.

[Wauplin]

@WesleyTheGeolien I just shipped huggingface_hub==0.23.3 with the mentioned fix. Thanks again for noticing and reporting the error :hugs:

[WesleyTheGeolien]

@Wauplin 0.23.1 or 0.23.3?? https://github.com/huggingface/huggingface_hub/releases/tag/v0.23.3 this is the correct one right?

[Wauplin]

Yes indeed sorry. Corrected my typo :)

[WesleyTheGeolien]

Confirmed it is also working thanks for the speedy fix!