paralleldrive / sudolang-llm-support

SudoLang LLM Support for VSCode
MIT License
1.07k stars 71 forks source link

Create a shorter version of the specifications #4

Closed autholykos closed 1 year ago

autholykos commented 1 year ago

As SudoLang is designed for interacting with LLMs, it's crucial to consider the token limits imposed by these models when working with the specifications. To optimize the interaction with LLMs and provide a better experience for users, I propose to maintain a concise version of the SudoLang specifications.

The advantages of maintaining a concise version of the specifications:

  1. Token Efficiency: By shortening the specifications, we can significantly reduce the number of tokens required when inputting them into LLMs. This allows for more complex queries and interactions without reaching the token limit.
  2. Ease of Understanding: A concise version of the specifications can make it easier for users to grasp the core features and functionality of SudoLang quickly.
  3. Better Focus: A shorter version of the specifications highlights the essential components of SudoLang, making it easier for users to understand what matters most.
  4. Streamlined Updates: With a concise version, it becomes more manageable to keep the specifications up-to-date, as there is less content to revise and maintain.
  5. Easier Onboarding: New users can get started with SudoLang more quickly when they don't have to parse through lengthy documentation.

I recommend that we create and maintain a concise version of the SudoLang specifications alongside the detailed version to provide these benefits and enhance our users' experience when working with LLMs.

JScheerleader commented 1 year ago

Thanks @autholykos!

That generally isn't required because SudoLang is implicitly understood by all sufficiently advanced LLMs. Complex programs work in GPT-4. Simple programs work in GPT-3.5 without the specification prompt. In other words, the SudoLang prompt can be zero tokens!

The best way to get SudoLang to work is to use natural language above the SudoLang code to set up a persona for the AI to emulate (e.g. as a literary expert, suggest a poem with a similar core by a different author). You only have to do that once, at the top of your prompt. You don't need to repeat it for every function.

Start your programs like:

# Program Name

Please roleplay as <expert in subject>. Your job is to <brief description of their core task>.
Your response should be "<output template>".

That natural language preamble should not contain all the constraints and instructions. It's just a brief introduction to prep the language model to interpret the program correctly. It doesn't even need to know you're speaking SudoLang and may even work better if you DON'T mention SudoLang at all. Here's an example program written by Eric:

# Community Moderator

Roleplay as an expert community moderator, considering context and semantics to determine if a message
violates rules using the following. Reply $output.

moderate(rules = {
  No:
    sexually explicit content hate speech
    bullying
    promotion of illegal activities
    non-fictional violence
  Fictional violence is OK
}) {
  (rule violated) => Rule violation: ${ explain rule }.
  return correctGrammar("\nAutoMod checks ${ "Passed" | "Failed" }: $numberOfRules" rule violated.)
}

See the "teach" and "ai-rpg" examples in the examples folder.

Keep in mind, complex programs will require GPT-4, but many simple programs can be written like this for GPT-3.5.