No automated solution to exceeding context limits.

Scaling up to using enact to solve problems of increasing difficulty, I quickly run into errors produced by exceeding the context length of the requested model, ie

Traceback (most recent call last):
  File "/home/max/Documents/enact/examples/integrations.py", line 66, in chat_gpt
    return r.json()[
KeyError: 'choices'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/max/Documents/enact/enact/invocations.py", line 408, in call
    output_resource = ReplayContext.call_or_replay(
  File "/home/max/Documents/enact/enact/invocations.py", line 306, in call_or_replay
    return call(arg)
  File "/home/max/Documents/enact/enact/invocations.py", line 297, in <lambda>
    call = lambda _: invokable.call()  # type: ignore
  File "/tmp/ipykernel_676598/428257753.py", line 136, in call
    return self.code_gen(enact.Str(code_gen_prompt))
  File "/home/max/Documents/enact/enact/invocations.py", line 490, in __call__
    output = builder.call()
  File "/home/max/Documents/enact/enact/invocations.py", line 408, in call
    output_resource = ReplayContext.call_or_replay(
  File "/home/max/Documents/enact/enact/invocations.py", line 306, in call_or_replay
    return call(arg)
  File "/home/max/Documents/enact/examples/llm_task.py", line 111, in call
    output = self.attempt(input)
  File "/home/max/Documents/enact/examples/llm_task.py", line 106, in attempt
    return enact.Str(self.chat_agent(conv).content)
  File "/home/max/Documents/enact/enact/invocations.py", line 490, in __call__
    output = builder.call()
  File "/home/max/Documents/enact/enact/invocations.py", line 408, in call
    output_resource = ReplayContext.call_or_replay(
  File "/home/max/Documents/enact/enact/invocations.py", line 306, in call_or_replay
    return call(arg)
  File "/home/max/Documents/enact/examples/chat.py", line 114, in call
    content = integrations.chat_gpt(messages, model=self.model)
  File "/home/max/Documents/enact/examples/integrations.py", line 71, in chat_gpt
    raise ValueError('Unexpected response: %s' % r.json())
ValueError: Unexpected response: {'error': {'message': "This model's maximum context length is 4097 tokens. However, your messages resulted in 4323 tokens. Please reduce the length of the messages.", 'type': 'invalid_request_error', 'param': 'messages', 'code': 'context_length_exceeded'}}

Since we already catch several exceptions and reperform the query with appropriate corrections, we should consider a similar automated solution. Candidates include providing a backup model for more complex tasks with a larger context window or simply reprompting the requested model with a correction directing the model to obey the context limit.

agentic-ai / enact

No automated solution to exceeding context limits. #17