Remote Execution Exception: Unprocessable Entity

Hello NDIF Team,

I was following the tutorial https://nnsight.net/notebooks/tutorials/walkthrough/#2-Bigger to learn nnsight with remote execution. After registering the API key and trying to reproduce the tutorial code, I got an Exception: Unprocessable Entity. May I ask how to deal with this issue? I basically followed the tutorial step by step. Was it because the API key wasn't properly set up?

Code:

# We'll never actually load the parameters so no need to specify a device_map.
model = LanguageModel("meta-llama/Llama-2-70b-hf")

# All we need to specify using NDIF vs executing locally is remote=True.
with model.trace("The Eiffel Tower is in the city of", remote=True) as runner:

    hidden_states = model.model.layers[-1].output.save()

    output = model.output.save()

print(hidden_states)

print(output["logits"])

Error

---------------------------------------------------------------------------
Exception                                 Traceback (most recent call last)
Cell In[27], line 5
      2 model = LanguageModel(\"meta-llama/Llama-2-70b-hf\")
      4 # All we need to specify using NDIF vs executing locally is remote=True.
----> 5 with model.trace(\"The Eiffel Tower is in the city of\", remote=True) as runner:
      7     hidden_states = model.model.layers[-1].output.save()
      9     output = model.output.save()

File /opt/anaconda3/lib/python3.12/site-packages/nnsight/contexts/Runner.py:44, in Runner.__exit__(self, exc_type, exc_val, exc_tb)
     41     raise exc_val
     43 if self.remote:
---> 44     self.run_server()
     46     self._graph.tracing = False
     47     self._graph = None

File /opt/anaconda3/lib/python3.12/site-packages/nnsight/contexts/Runner.py:61, in Runner.run_server(self)
     53 request = pydantics.RequestModel(
     54     kwargs=self._kwargs,
     55     repo_id=self._model._model_key,
     56     batched_input=self._batched_input,
     57     intervention_graph=self._graph.nodes,
     58 )
     60 if self.blocking:
---> 61     self.blocking_request(request)
     62 else:
     63     self.non_blocking_request(request)

File /opt/anaconda3/lib/python3.12/site-packages/nnsight/contexts/Runner.py:146, in Runner.blocking_request(self, request)
    142     response = pydantics.ResponseModel(**response.json())
    144 else:
--> 146     raise Exception(response.reason)
    148 remote_logger.info(response)
    150 while True:

Exception: Unprocessable Entity

ndif-team / nnsight

Remote Execution Exception: Unprocessable Entity #191