paralleldrive / sudolang-llm-support

SudoLang LLM Support for VSCode
MIT License
1.07k stars 71 forks source link

Pseudo-compilation? #3

Closed miguelwon closed 1 year ago

miguelwon commented 1 year ago

Not really an issue. Just to comment that this repo is really amazing. Thanks for sharing! What about a kind of pseudo-compilation to help debug the code itself?

After a few trials got this prompt:

You are an AI Assistant ruled by the program shown bellow. The program is written in SudoLang, a pseudo-programming language created to interact programatically with LLMs. SudoLang code abstract instructions, written in natural language, to be interpreted by the LLM. Just like other running programming languages you need to first read the instructions and simulate a psedo-compilation process to check inconsistencies such as syntax errors. Print the compilation errors if it is the case. Otherwise just run the code. Don't print unnecessary observations.

That used with this example:

The program: 

function square: (x){
   return x times x times x
}
function askAndWaitForReply: {
   reply = ask("Give me a number")
   a,b = square(reply
   return "$a^2 = b"
}

askAndWaitForReply()

got this reply from GPT-4:

Compilation errors:

In function 'square', the calculation should be x times x, not x times x times x.
In function 'askAndWaitForReply', there is a missing parenthesis in the line 'a,b = square(reply'.
Corrected program:

function square: (x){
return x times x
}
function askAndWaitForReply: {
reply = ask("Give me a number")
a, b = square(reply)
return "$a^2 = b"
}

askAndWaitForReply()

And then running with the proposed version still got a new error:

There is a syntax error in the program. The function square should return only one value, but two values (a, b) are being assigned when calling the square function. Here's the corrected version of the program:

function square: (x){
return x times x
}
function askAndWaitForReply: {
reply = ask("Give me a number")
b = square(reply)
return "$reply^2 = b"
}

askAndWaitForReply()

Now running the code:

Give me a number: 5
5^2

And then I changed finally to the program:

The program:

function square: (x){
return x, x times x
}
function askAndWaitForReply: {
reply = ask("Give me a number")
a, b = square(reply)
return "$a^2 = $b"
}

askAndWaitForReply()

It resulted in:

GPT-4: Give me a number
Me:  7
GPT-4: 7^2 = 49
ericelliott commented 1 year ago

SudoLang has a built-in linter.