evangriffiths
commented
1 month ago Here's the commit where the training feature is added https://github.com/crewAIInc/crewAI/commit/175d5b3dd6a37b54708a89bcca64df2122f2b7c9.
My understanding of how it works:
- it builds on top of an existing 'human feedback' feature that crewai has, where, if enabled:
- when an agent has finished its task, it gives the proposed output to the user, who can give feedback
- the agent re-runs with this feedback to produce the final output
crew.trainturns this into an iterative process, where it runs the agent normally, with the 'human feedback' feature enabled, but...- each iteration the proposed output, the human feedback, and the final output are appended to a .pkl file
- and each iteration this is loaded and appended to the prompt for each agent
- then at the end of the training process, the contents of the .pkl file (the "training data") for each agent are sythesized/summarized into useful learnings, and these are saved to a separate .pkl file
- now when the crew is run normally, outside of the
trainprocess, it loads the contents of the second .pkl file into the agent's prompt
All-in-all, this is quite similar to what we were trying to do with RememberPastActions, but with important differences:
- it uses human-in-the-loop to evaluate agent actions -- we don't have any explicit functionality atm for the agent to evaluate its strategy.
- it is way more structured -- i.e. we just let the agent decide when to do
RememberPastActions, but here the human gives feedback every iteration, and this is appended to the prompt in a fixed way.
I haven't tried crewai.train, so don't know if it works that well, but I think we could take inspiration from this for the general agent for sure.
https://docs.crewai.com/core-concepts/Training-Crew
Have only had a glance so far. Looks like it does some form of RL to tune the prompts! This is exactly the kind of feature we want for our PM agents to learn over time!