Using the latest pypi release (pip install dspy-ai==2.1.9), vLLM Client imports and works fine:
>>> import dspy
>>> lm = dspy.HFClientVLLM(model="NousResearch/Llama-2-7b-chat-hf", port=8000, url="http://localhost")
>>> lm("My name is ")
["👀 and I'm a 🐶. I'm a very good boy and I love to play fetch and go on long walks. I'm also very good at giving hugs and snuggles. I'm a bit of a foodie and I love treats, but I'm also happy to eat my kibble. I'm a bit of a cuddlebug and I love to be close to my humans. I'm a bit of a goofball and I love to play and have fun. I'm a very good boy and I'm looking for a forever home where I can be loved and adored. 🐾"]
However with the latest main this fails on import:
Using the latest pypi release (
pip install dspy-ai==2.1.9
), vLLM Client imports and works fine:However with the latest main this fails on import: