AnthusAI / Plexus

An orchestration system for managing text classification at scale using LLMs, ML models, and NLP.
https://anthusai.github.io/Plexus/
MIT License
4 stars 0 forks source link

Does `train_model()` make sense for `AgenticValidator`? #6

Closed endymion closed 1 month ago

endymion commented 2 months ago

There is a train_model() function in the AgenticValidator class, but does that make sense?

This needs better documentation, but train_model() is a standard lifecycle function for a Plexus Score that's vital for ML models that require training weights or whatever and then saving a model artifact file for use during inference. But the AgenticValidator operates on retrained models through APIs, and doesn't require doing anything like that at training time, as far as I know?

Is that functionality in the function right now important for something? I'm worried that it will never be invoked.

We probably need a standard document_score() lifecycle function for Plexus Scores that will generate model diagrams for ML models graph diagrams for LangGraph agentic Scores, etc.

dereknorrbom commented 2 months ago

The base class required it. I brought this up previously but didn't get an actionable response.

https://github.com/AnthusAI/Plexus/commit/092e04415b8dcb299472345f4697e73744713f06

I implemented a pass on train_model and gave the method a more appropriate name create_validation_agent.

endymion commented 1 month ago

Ah, okay, that makes sense. I was just worried about errors from that function getting called when we call plexus train ... on a scorecard with a mixture of those scores and ML scores.