Dynamic prompting for ai_classifier

[dleebrown]

First check

• [X] I added a descriptive title to this issue.
• [X] I used the GitHub search to look for a similar issue and didn't find it.
• [X] I searched the Marvin documentation for this feature.

Describe the current behavior

Currently the @ai_classifier decorator can only decorate an function with Enum with no extra parameters.

Describe the proposed behavior

Add in the ability to dynamically change the classification instructions for ai_classifier, i.e. have the ability to provide a prompt at function runtime that instructs how to evaluate an input string for class membership.

Example Use

Here's a toy example:

@ai_classifier
class RelevantSite(Enum, user_need):
    """Whether or not the provided website is relevant to the user"""
   YES = 1
   NO = 0

RelevantSite("www.google.com", "I want a search engine")

Additional context

I believe ai_function has the same limitations on dynamic task prompting.

Currently the workaround I'm using is to modify the docstring e.g. RelevantSite.__doc__ += user_need, which works, but having a first-party implementation would be nice.

[aaazzam]

Hey @dleebrown. Is the DX you're looking for something like:

@ai_classier
class Sentiment(Enum):
    '''Classifier to determine whether someone {{descriptor}} thinks the joke is good or bed.''
    GOOD = 0
    BAD = 1

Sentiment("My choral teacher's startup just got a'choir-ed", descriptor = "who loves puns")
# >> Sentiment.GOOD
Sentiment("My choral teacher's startup just got a'choir-ed", descriptor = "who hates puns")
# >> Sentiment.BAD

Or are you looking more for

from marvin import AIEnum

class Sentiment(AIEnum):
    prompt: Prompt

    GOOD = 0
    BAD = 1

Sentiment("My choral teacher's startup just got a'choir-ed", **context)...

Or is there a different DX that would make your live easier? I know you mentioned Enum's not being your go-to object.

taggin @jlowin and @zzstoatzz for comment

[dleebrown]

I know we talked about this already and I waffled then, but I think I like the jinja-esque templating because it a) is super frictionless to not use (i.e. static prompting with docstrings is no problem/either path feels intuitive to me at least) and b) doesn't require creation of a novel data type (AIEnum) despite my general meh ness about enums.

[jlowin]

Welcome aboard @dleebrown!

To skip to the punchline I've really been liking the instructions="" pattern we've used on AI Models and AI Functions because its clear and discoverable to users, and would encourage it here:

@ai_classier
class Sentiment(Enum):
    '''Classifier to determine whether someone thinks the joke is good or bed.''
    GOOD = 0
    BAD = 1

Sentiment("My choral teacher's startup just got a'choir-ed", instructions="consider a person who loves puns")
# >> Sentiment.GOOD
Sentiment("My choral teacher's startup just got a'choir-ed", instructions="consider a person who hates puns")
# >> Sentiment.BAD

The reason I prefer this over jinja templating is that I think jinja is extremely useful for us internally to build up prompts from known variables but very difficult to expose to users -- even discovering that the keyword is "descriptor" would be a challenge in the above proposal, let alone using it. The code also fails to be self documenting unless you have the source code of the classifier right in front of you (and even then you have to scan for {{'s)

The benefit of instructions= is that its clear 1) that this is a freeform instruction 2) that the audience of the instruction is the AI Classifier itself (in a sense) and 3) it's documentable and standard. If someone really wanted to recreate the situation with the templated variable in the middle of the string, they are welcome to design their classifier to do their own instructions templating however they like.

[dleebrown]

👋 happy to be here!

The points around having a simple freeform instruction that gets passed in makes sense to me, esp. if there's no requirement of a novel data type.

Adopting this pattern - how should I be using the docstring in the function vs. the instructions that get passed in? Is the correct framing that the docstring more or less defines abstract classification task, while the instructions define how to treat the accompanying Enum? Are the two fields degenerate to some extent, e.g. as a facetious example I could do:

@ai_classier
class Sentiment(Enum):
    '''consider a person who loves puns''
    GOOD = 0
    BAD = 1

Sentiment("My choral teacher's startup just got a'choir-ed", instructions="Classifier to determine whether someone thinks the joke is good or bed.")
# >> Sentiment.GOOD

FWIW I had similar confusion at times around the difference between instruction and personality and prompt in the old Bot in pre-1.0-marvin, so my confusion may just be a RTFM type situation here.

[jlowin]

Here, I would consider both the docstring AND the instructions cooperating to form the system prompt, and the docstring is the model author's opportunity to define its behavior, and the instructions are the model caller's opportunity to define behavior. In a sense the docstring contains class behavior and the instructions specify instance behavior. For example, if I publish a translation model, I might say:

@ai_model
class Translation(BaseModel):
    """Translate from provided text to another language"""
    original_text: str
    translated_text: str

Now I could write that this model translates from English to say, French, in its docstring, but then I'm declaring that its really a FrenchTranslation model. Alternatively, I could set the target language at runtime using instructions over this more generic class, e.g. Translation("Hello", instructions_='translate to french'); Translation("Hello", instructions_='translate to german')

With that in mind I would consider your Sentiment class valid in the sense that it would work but probably not good practice, since the instructions should be used to "steer" the more general docstring instruction rather than the other way around.

[jlowin]

In good faith the pure "class vs instance" distinction of the docstring vs the instructions breaks down because you can provide instructions to the ai_model decorator that get automatically passed to each instance (but are common to every instance). In practice I think this construction is confusing because of the redundancy and would avoid it in favor of just a docstring or just decorator-level instructions.

@ai_model(instructions="translate to french")
class Translation(BaseModel):
    """Translate from provided text to another language"""
    original_text: str
    translated_text: str

[jlowin]

If you're testing this ^ there's a small bug where instructions are added to the global prompts list so subsequent calls get confused. Will have a fix shortly.

[aaazzam]

Implementation to address this is added in #470 @dleebrown

[aaazzam]

Fixed in 1.3