Open scenaristeur opened 1 month ago
to avoid this error, i have changed agent.py line 710 to use self.summarize_messages_inplace() . and now it works . it seems that the messages send were too long
except Exception as e:
printd(f"step() failed\nuser_message = {user_message}\nerror = {e}")
# If we got a context alert, try trimming the messages length, then try again
#if is_context_overflow_error(e):
# A separate API call to run a summarizer
self.summarize_messages_inplace()
# Try step again
return self.step(user_message, first_message=first_message, return_dicts=return_dicts)
#else:
# printd(f"step() failed with an unrecognized exception: '{str(e)}'")
# raise e
to avoid this error, i have changed agent.py line 710 to use self.summarize_messages_inplace() . and now it works . it seems that the messages send were too long
except Exception as e: printd(f"step() failed\nuser_message = {user_message}\nerror = {e}") # If we got a context alert, try trimming the messages length, then try again #if is_context_overflow_error(e): # A separate API call to run a summarizer self.summarize_messages_inplace() # Try step again return self.step(user_message, first_message=first_message, return_dicts=return_dicts) #else: # printd(f"step() failed with an unrecognized exception: '{str(e)}'") # raise e
did you write a PR for this? .. the free inference.memgpt.ai made available since the beginning I really appreciate as do others with limited gpu/ compute power .. but the experience you describe has been going on since the beginning. please make a pr - Thanks for finding a pos solution .. will be testing it out
wow this little fix really worked! Thanks!
Hi @quantumalchemy thxs for your feedback of my fix. ;-)
it seems that the only error catched making a summarize is "if is_context_overflow_error(e):" but the inference process return a 404/500 error, when a too long message is send to the inference server that is not catched. this way if i get a 400/500 error i try a summarization and retry the step. but if it is another error it does it too, so could possibly loop with infinity run and potentialy infinity api consumption if we does not take care. perharps @sarahwooders or @cpacker could find a better way to summarize, or treat this 400/500 errors that are in fact context overflow errors
thanks again for figuring this out .. I think your patch here only works for the memgpt llm endpoint so I created a fork .. testing it works on latest v. + patch was getting --> Failed to put inner thoughts in kwargs: Invalid control character at: line 2 column 270 (char 271) without patch -- but like you said .. goes a little slower because of some extra looping.. but knock wood -- not on error.
I have tried many conversation with memGPT agent using https://inference.memgpt.ai .
It sounds good, at the beginning But at some point of the conversation, after about 10 or 20 exchange between user and agent, it falls with a 4xx/5xx error
If i start another conversation from scratch, it is ok at the beginning but crashes same way at some point. Is there a limitation of query per minute ( i don't think so , because if i wait for 2 min , my conversation is broken, and i can get something else than this error)
here is my config
and here is the crash