paralleldrive / sudolang-llm-support

SudoLang LLM Support for VSCode
MIT License
1.07k stars 71 forks source link

How to output only code for scripting? #11

Closed joshmedeski closed 1 month ago

joshmedeski commented 10 months ago

I'm enjoying using Sudolang, thanks for the spec!

I'm wondering how do you coerce Sudolang to only output code without any markdown or code blocks?

# ColorPaletteGenerator

Generate color palettes for terminal emulators.
Your job is to generate smart color palettes that are cohesive and pleasant looking.
You will output into different formats to work with different terminal emulators.

Here is an example alacritty color palette:
```yaml
# Alacritty format
colors:
  # Default colors
  primary:
    background: "0x181818"
    foreground: "0xd8d8d8"

  # Colors the cursor will use if `custom_cursor_colors` is true
  cursor:
    text: "0x181818"
    cursor: "0xd8d8d8"

  # Normal colors
  normal:
    black: "0x181818"
    red: "0xab4642"
    green: "0xa1b56c"
    yellow: "0xf7ca88"
    blue: "0x7cafc2"
    magenta: "0xba8baf"
    cyan: "0x86c1b9"
    white: "0xd8d8d8"

  # Bright colors
  bright:
    black: "0x585858"
    red: "0xab4642"
    green: "0xa1b56c"
    yellow: "0xf7ca88"
    blue: "0x7cafc2"
    magenta: "0xba8baf"
    cyan: "0x86c1b9"
    white: "0xf8f8f8"

ColorPalette { State { Background Foreground Black White Red Magenta Yellow Green Blue Cyan } Constraints { You will generate a color palette.

Instruct the AI:
- Colors should all feel cohesive
- Output in the Alacritty color scheme format in yaml
- Always output raw, never use markdown
- Never explain the answer
- Remove codeblocks

}

function transpile { ColorPalette |> transpile(yaml) }

/transpile colors? }



I want the output to be in the specific format from the example code, but I'm having a hard time figuring out how to get OpenAI to only output the code and nothing else (no markdown code blocks, no messages explaining the response).

How does this work? I wrote a bash script and I want to dump the output to a file to auto-update my configuration.

Thanks for the help.
dinh commented 10 months ago

How about this?

`Roleplay as a shell expert with great inowledge in color palettes for emulators.

PaletteGen { State { Target system: Linux, Mac OS } Constraints {

joshmedeski commented 10 months ago

Nope, I'm getting pretty unreliable output with Sudolang. Sometimes it says it can't complete the task, other times it creates python code for me to use to generate an Alacritty theme, and other times it output an Alacritty theme with an explanation and a markdown code block. And rarely I get the output I'm hoping for (only yaml).

I'm guess it's just not clicking for me.

I learned and experimented with Sudolang in my live stream recently if you're curious: https://www.youtube.com/watch?v=pAgvtMiV5ZI

ericelliott commented 1 month ago

Check out the prompt for the robot example here: https://github.com/paralleldrive/sudolang-llm-support/blob/main/examples/robot.sudo

It does a pretty good job of reliably delivering responses in the requested format.

Prompting is a skill that takes practice. I have been prompting with pseudocode since the summer of 2020, and I still get frustrated from time to time.

The model you use also matters a lot. GTP-4o is not very good at following instructions. The best model I'm aware of right now is Claude 3.5 Sonnet.

joshmedeski commented 1 month ago

Thanks for the feedback @ericelliott

I agree prompting is a practiced skill! It's been almost a year since I opened this ticket and I'm still learning.