Exception: Received invalid JSON response from server

[andysingal]

# install DSPy: pip install dspy
import dspy

# This sets up the language model for DSPy in this case we are using mistral 7b through TGI (Text Generation Interface from HuggingFace)
mistral = dspy.HFClientTGI(model='mistralai/Mistral-7B-v0.2', port=8080, url='http://localhost')

# This sets the language model for DSPy.
dspy.settings.configure(lm=mistral)

# This is not required but it helps to understand what is happening
my_example = {
    "question": "What system was final fantasy 1 made for?",
    "answer": "NES",
}

# This is the signature for the predictor. It is a simple question and answer model.
class BasicQA(dspy.Signature):
    """Answer questions with short factoid answers."""

    question = dspy.InputField()
    answer = dspy.OutputField(desc="often between 1 and 5 words")

# Define the predictor.
generate_answer = dspy.Predict(BasicQA)

# Call the predictor on a particular input.
pred = generate_answer(question=my_example['question'])

# Print the answer...profit :)
print(pred.answer)

gives error:

```py
Failed to parse JSON response: 
---------------------------------------------------------------------------
JSONDecodeError                           Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/requests/models.py](https://localhost:8080/#) in json(self, **kwargs)
    970         try:
--> 971             return complexjson.loads(self.text, **kwargs)
    972         except JSONDecodeError as e:

12 frames
JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

JSONDecodeError                           Traceback (most recent call last)
JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Exception                                 Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/dsp/modules/hf_client.py](https://localhost:8080/#) in _generate(self, prompt, **kwargs)
     96         except Exception:
     97             print("Failed to parse JSON response:", response.text)
---> 98             raise Exception("Received invalid JSON response from server")
     99 
    100 

Exception: Received invalid JSON response from server

[mikeedjones]

If you call the model directly, not using dspy, do you get a valid response?