Warning from `requests` package when using any PyTorch application

[ad8e]

Testcase:

1. clone https://github.com/pytorch-labs/gpt-fast
2. Run this in the gpt-fast folder

   export MODEL_REPO=openlm-research/open_llama_7b
   ./scripts/prepare.sh $MODEL_REPO
   python generate.py --compile --checkpoint_path checkpoints/$MODEL_REPO/model.pth --prompt "Hello, my name is"

Result:

/usr/lib/python3/dist-packages/requests/__init__.py:89: RequestsDependencyWarning: urllib3 (1.26.18) or chardet (3.0.4) doesn't match a supported version!
  warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

I solved this by running pip3 install --upgrade requests, which is the solution google gives.

Zero urgency; our image already fixes this, so it's not affecting anything.

[Eta0]

Which container image (name and tag) are you using from this repository? When I looked, I couldn't find any from the latest releases of torch or torch-extras with urllib3 1.26 installed.

[ad8e]

ghcr.io/coreweave/ml-containers/torch-extras:73a87a6-nccl-cuda12.2.2-ubuntu22.04-nccl2.19.3-1-torch2.2.0-vision0.17.0-audio2.2.0-flash_attn2.4.2

[Eta0]

The path in your error message (/usr/lib/python3/dist-packages/requests/__init__.py) does not exist in that container image. That image has the following package versions:

• requests==2.31.0
  • Newest version of requests available
  • Requires urllib>=1.21.1 & (chardet>=3.0.2,<6.0.0 or charset_normalizer>=2.0.0,<4.0.0)
  • Is too new to produce the error message shared
• urllib3==2.2.0 (compatible)
• No chardet
• charset_normalizer==3.3.2 (compatible)

Your error message reports an older version of requests, a different version of urllib3 (1.26.18), and a different version of chardet (not present in that image), so it doesn't seem to be our container image causing those version incompatibilities.

The script to reproduce the issue does not run on that container image out of the box—it fails because huggingface_hub and sentencepiece are not installed, so you're installing more dependencies at some point. With that in mind, it may be your workflow that is overriding some part of the container image with incompatible packages during one such installation step, and that would need to be fixed on your side.

[ad8e]

You're right, it's almost certainly my own packages causing the issue. Sorry for the noise.