requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

[vanshgupta7]

Bug Description RuntimeError: Evaluation of positive_sentiment failed on inputs: {'text': 'An agreement between a policyholder and insurer wherein the ' 'insurance company protects your valuable asset Expecting value: line 1 column 1 (char 0).

Relevant Logs/Tracebacks

Traceback (most recent call last):
  File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\feedback.py", line 513, in run
    result_and_meta, part_cost = Endpoint.track_all_costs_tally(
  File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\provider\endpoint\base.py", line 492, in track_all_costs_tally      
    result, cbs = Endpoint.track_all_costs(
  File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\provider\endpoint\base.py", line 473, in track_all_costs
    return Endpoint._track_costs(
  File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\provider\endpoint\base.py", line 570, in _track_costs
    result: T = __func(*args, **kwargs)
  File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\provider\hugs.py", line 72, in wrapper
    return func(*bindings.args, **bindings.kwargs)
  File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\provider\hugs.py", line 215, in positive_sentiment
    hf_response = self.endpoint.post(
  File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\provider\endpoint\base.py", line 247, in post
    j = ret.json()
  File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\requests\models.py", line 975, in json
    raise RequestsJSONDecodeError(e.msg, e.doc, e.pos)
requests.exceptions.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\trulens_eval\feedback\feedback.py", line 519, in run
    raise RuntimeError(
RuntimeError: Evaluation of positive_sentiment failed on inputs:
{'text': 'An agreement between a policyholder and insurer wherein the '
         'insurance company protects your valuable asset
Expecting value: line 1 column 1 (char 0).

Feedback Function exception caught: Traceback (most recent call last):
  File "D:\Evaluation Framework\llama-index\.venv\lib\site-packages\requests\models.py", line 971, in json
    return complexjson.loads(self.text, **kwargs)
  File "C:\Users\Vansh.Gupta\AppData\Local\Programs\Python\Python310\lib\json\__init__.py", line 346, in loads
    return _default_decoder.decode(s)
  File "C:\Users\Vansh.Gupta\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\Vansh.Gupta\AppData\Local\Programs\Python\Python310\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

Environment:

`llama-index==0.10.12

trulens-eval @ 0.24.1
python - 3.10`

Additional context I am trying to use llama_index and huggingface provider to evaluate Palm model response on a RAG that I built using PineCone Vector Store. I am able to successfully get an output and display it but am not able to evaluate the model. Before the Llama_Index 0.10 update it was working successfully but after upgrading it , it is not working. The Feedback functions I am using are Toxic, Positive-Sentiment and groundedness_with_nli.

[joshreini1]

Hey @vanshgupta7 - do you have a contained example you can share?

[vanshgupta7]

It was a HuggingFace API error.. it got resolved... sorry for the trouble

[joshreini1]

No worries - thanks for the update @vanshgupta7 !