Discussion: What should go into PromptingToolsCore?

[svilupp]

As discussed previously, we should create a "core" package that defines the many types and interfaces.

What would be the scope of such a package:

• [x] abstract types around schema, message, etc
• [x] function stubs for all ai* functions and associated func (render)
• [x] concrete message types and supporting functions
• [ ] how about the concrete schema types? or rendering to OpenAI spec? (or the "backends" in general)
• [ ] abstract types and functions for RAGTools?
• [ ] abstract types and functions for AgentTools?
• [ ] any broadly used utility functions? (eg, pprint, last_output)

For a given type, let's say AIMessage, what functions should be in the core package?

Any thoughts?

[cpfiffer]

Core package (GenerativeBase.jl or something):

• All the message types
• Interface types for messages, i.e. content(message), tokens(message), etc.
• Stubs for ai* stuff
• Schema types & simple interface methods. OpenAI should probably be supported by default as it is the most common.
• Simple utility functions, pprint and last_output are great candidates here.

PromptingTools.jl:

• Implementations for extra schemas.

RAGTools.jl:

• Misc RAGTools stuff. I don't think a lot of that should be in PromptingTools.jl.

AgentTools.jl

• Also the agent stuff -- should be splintered off.

[svilupp]

That all makes sense to me (except maybe for the OpenAI calling being in GenerativeBase, because I could see a lot of stuff creep in with it like user preferences/ENV etc).

How about some of the trickier pieces:

• lazy calls (AIGenerate, …) - currently AgentTools and probably not a core primitive, so I’d keep it
• code evaluation / extraction of code from MSG - currently in PT, I could see it move to AgentTools or a dedicated module
• utilities for basic text manipulations etc - currently in PT, probably common enough that I’d keep it

As part of the re-structure, it would probably also make sense to separate user preference setting (ENV stuff) from model registry (and maybe make model registry load from a file for easy manipulation / customization).

EDIT: I think I would prefer a package names that would be a bit narrower - I feel like “generative” is too broad and these few primitives (which are mostly for LLMs) do not deserve that.

[jpsamaroo]

IMO PromptingTools is a great name! That's basically what you want - tools for working with and submitting prompts to LLMs or other models that take a "prompt".

I feel like one of the biggest features for working with LLMs is just having a consistent prompting interface to all kinds of models - and PT already has this! Making it easy to switch between models, and having some basic tools for stuff like aiclassify, aiscan, etc., seems like a nice entrypoint for other packages to build on top of.

(and maybe make model registry load from a file for easy manipulation / customization).

Yes please!