No "Run GenAIScript" option in the right-click context menu after running pdf parser on Windows

Type: Bug

Describe the issue

I have script that parses a pdf file. I ran it on a pdf file and it ran successfully but when I try to invoke a script on another pdf file, no option appears in the right-click context menu

It looks like I'm in the debugger when the script ends.

To Reproduce

See above

Expected behavior

I expected to see the option

Environment

vscode: 1.88.1 extension: 1.19.2

Request

## gen-tweet

<details id="🤖-automation" class="genaiscript">
<summary>
🤖 automation
</summary>

This operation can be automated using the command line interface.

- to run on all files at once, use the `run` command:

```bash
npx --yes genaiscript@^1.19.2 run gen-tweet "GenAIScript-Introduction-March2024.pdf" --apply-edits

to run a LLM generation on each file, use the batch command:

npx --yes genaiscript@^1.19.2 batch gen-tweet "GenAIScript-Introduction-March2024.pdf" --apply-edits

You will need to install Node.js.
The CLI uses the same secrets in the .env file.
Run npx genaiscript help run for the full list of options.

📄 spec

```markdown # Specification - [GenAIScript-Introduction-March2024.pdf](./GenAIScript-Introduction-March2024.pdf) ```

💾 script

🧬 prompt

📓 script source

```js script({ title: "gen-tweet" }) def("FILE", env.files) $`Given the paper in FILE, write a 140 character summary of the paper that makes the paper sound exciting and encourages readers to look at it. ` ```

- model: gpt-4 - temperature: 0.2 - tokens: 4129 ````markdown FILE: ```pdf file="GenAIScript-Introduction-March2024.pdf" GenAIScript: Robust Scripting for Generative AI Peli de Halleux, Markus Kuppe, Michał Moskal, Madan Musuvathi, Benjamin Zorn March 2024 Abstract our existing computing infrastructure, AISW, with its enhanced capabilities, will dominate the development Scripting languages, like Python and JavaScript, have of future software systems. Our goal is to make enabled major advances in computing but were created GenAIScript the language for scripting AISW. prior to the advent of Large Language Models (LLMs). This paper introduces GenAIScript, a script- Historically, scripting languages like Perl, JavaScript, ing language that makes LLMs a first-class part of the and Python, have become widely successful because scripting process, allowing users to author, debug, and they provide two key elements: (1) they allow diverse deploy LLM-based scripts that can perform tasks be- users to write small but useful programs, and (2) they yond the reach of conventional code. By extending the give users access to new computing capabilities not popular Visual Studio Code environment, GenAIS- previously available to programs (e.g., JavaScript gave cript simplifies the creation of robust and reusable its users access to the Web). scripts that can intelligently process and generate con- Given that context, the goal of GenAIScript is to em- tent, including code, documentation, and structured power a wide class of potential users to easily leverage data. We explain what GenAIScript is, how it is im- the power of AISW. GenAIScript enables users, in- plemented, and illustrate its use in several production cluding those without extensive programming exper- deployments. tise, to author, debug, and deploy scripts that seam- lessly incorporate calls to LLMs. GenAIScript is a Ja- 1. Introduction vaScript-compatible language with a dedicated VS Code extension to simplify the scripting process, mak- 1 GenAIScript is a JavaScript-compatiblescripting lan- ing it accessible to a broader audience. guage that allows users to write scripts to automate complex tasks by leveraging LLMs as a first-class With the goal of making GenAIScript the go-to lan- script element. guage for writing AISW, we have the following key objectives: Commented [BZ1]: @Madan Musuvathi - I added a list The dramatic rise in the capabilities of artificial intel- of key objectives and how we address them related to your ligence has revolutionized the way we interact with 1. Integration with LLMs: GenAIScript is de- question. Let me know what you think. technology, offering unprecedented opportunities to signed to make LLMs a first-class part of the automate and enhance many tasks. Incorporating calls scripting process, allowing users to author, to LLMs (or more generally any foundation model) at debug, and deploy LLM-based scripts. runtime into a software system greatly enhances what that software can do. The disruptive nature of this 2. Simple Abstractions: GenAIScript provides transition forces us to categorize software into Plain abstractions to define the prompt context, re- Ordinary Software (POSW), which does not use the fer to the context in the prompt itself, and power of LLMs at runtime, and the more capable AI supports specification and checking of multi- Software (AISW) that does leverage LLMs. ple output formats. POSW represents all software written before the crea- 3. Ease of Use: GenAIScript targets a wide tion of LLMs and includes operating systems, applica- class of potential users, including non-devel- tions, scripts, etc. While POSW is a key element of opers. The VS Code extension and 1 Currently GenAIScripts are JavaScript-compatible but in the future we anticipate bindings for other lan- guages including Python. 1 -------- Page Break -------- integration with existing IDE features con- Because every GenAIScript calls an LLM, metadata is tribute to this quality. needed to define the parameters that will be used to invoke the LLM. Those parameters can include the 4. Robustness: GenAIScript aims to be robust model, the temperature, the number of tokens to re- by leveraging the capabilities of VS Code for turn, etc. By default, GenAIScripts use the “gpt-4” writing, executing, debugging, and refining model and other reasonable defaults, so many of the scripts. Additional features like input and out- metadata fields can be omitted. put specifications and static orchestration graphs also contribute to robustness. A GenAIScript executes in three parts: 1) set up the LLM prompt, 2) execute the LLM, and 3) parse the 5. Seamless Integration with Plain Old Soft- LLM output. ware (POSW): GenAIScript is designed to be embedded into existing POSW work- Setting up the prompt is done with the “def” command, flows, allowing users to integrate AI capabil- which in this case associates the name FILES with the ities into traditional software systems. contents of env.files. env.files is a variable that refers to the parameters (typically one or more files) that the In the remainder of this paper, we describe the user ex- script is invoked on. After this def command, when the perience of creating a GenAIScript, define key user refers to FILES in the prompt, the LLM will know GenAIScript abstractions, highlight the capabilities of they are referring to the contents of the files in the framework, and describe several case studies based env.files. on our experiences building and deploying GenAIS- cripts in production setting. We discuss related work The text between $`…` in the figure is the prompt that and also our thoughts about future additions and fea- the LLM will process. The $`…` instructs the LLM what to do and refers to the names defined using def tures. such as FILE. This prompt is typically the final part of every GenAIScript. The prompt, augmented by any 2. Introducing GenAIScript content defined by defs, will be passed to the LLM. When the LLM returns a result, it will be either be pro- GenAIScript was designed to provide a low barrier to cessed by GenAIScript, or returned directly. entry so that many users, including non-developers, can quickly start using and writing scripts. However, This example, when executed, will shorten the con- we also want to enable power-users to leverage their tents of the input file as directed by the prompt. While programing skills to create and deploy more sophisti- the LLM generates a string as output, the GenAIScript cated scripts for more challenging scenarios. As a re- runtime understands standard output file formats, such sult, GeneAIScript is JavaScript compatible where the as code files (Python, etc.), markdown files, etc., so scripts can contain JavaScript code but less sophisti- that if the prompt instructs the model to produce a file cated users can still define powerful and effective of a given type, the runtime will parse the resulting scripts using the simple abstractions we provide. generation and create and populate the file. In this example, because the prompt says “shorten the following FILE”, the LLM will generate another file To understand GenAIScript, we present a simple com- with the same name. The GenAIScript runtime will plete script in Figure 1. create the new version of the file and then, using the existing refactoring feature that is part of VS Code, present the differences between the original file and the new file in the UI and ask the user to confirm the Figure 1: A simple GenAIScript This example illustrates the key elements of every Commented [BZ2]: @Peli de Halleux , @Markus Kuppe , GenAIScript: the metadata, the context, and the @Madan Musuvathi I added this description to the white prompt. We describe each element in turn. paper. Let me know what you think. 2 -------- Page Break -------- edits (shown in Figure 2). research but due to the uncertain nature of the quality and robustness of multi-agent interactions, in GenAIS- cript we have chosen a different approach toward the goal of creating more robust AISW. Specifically, in GenAIScript, when a script executes 4 there is a single call to the LLM. This approach al- lows the user to apply traditional techniques to im- prove their script’s software robustness (testing, etc.) leveraging many existing tools (debugging, refactor- ing, etc.) that are part of IDEs like VS Code. Every GenAIScript can be called from the command line. As a result, we encourage the integration of our Figure 2: Refactoring view of LLM output in GenAIScript scripts into existing POSW automation processes. We from script in Figure 1. believe that focusing the AISW on the specific hard tasks that POSW can’t do, and leveraging the enor- 3. GenAIScript Abstractions mous robust POSW infrastructure, is the best way to avoid known robustness challenges in building AISW. The design of GenAIScript follows several principles: As an aside, if a user writes a GenAIScript to generate - LLM at the bottom. By this we mean that a program as output, say in Python, it is not automati- every GenAIScript will use an LLM to cally executed. If the user wants to use GenAIScript achieve its goal. We firmly believe that if the to help write software by generating it automatically, user wants to accomplish something that is they still have to edit/execute/debug that software in a easily accomplished with POSW, they should traditional manner. 2 use that instead. Use GenAIScript for prob- lems that POSW cannot solve. Parameterization - Control AI Behavior with a Static Orches- tration Graph. We do not permit the LLM For scripts written in POSW, the parameters provide a in GenAIScripts to determine other AI way to generalize and customize the application of the scripts/agents to call and/or call them. In- script. For example, they might specify the name of stead, a user can determine statically exactly an input file so that the script can be applied to many what LLM will be called and with prompt different files. In general, for POSW, the input file context. does not affect how well the script itself executes or - Parameterizing context. We provide mech- what it does. anisms to bring external context into In GenAIScript, the input provided can have enormous GenAIScripts as well as provide mechanisms impact on what the script does and how well it does it. to structure that input to make it more com- This happens because inputs to the script are: prehensible to the LLM. - Specifying and verifying outputs. GenAIS- - Typically text files provided for Retrieval cript supports numerous ways to specify and Augmented Generation (RAG) check output requirements for the AI-gener- - Almost always passed to the underlying LLM ated content to ensure it conforms to the user to help it be more effective. specifications. For example, the prompt in a script might say: With these principles in mind, we discuss the key GenAIScript abstractions. Static Orchestration Graph Approaches to allowing a collection of AI agents to be defined and integrated have been proposed (for exam- 3 ple, autogen). This is an exciting area of current 2 4 Note that AI, such as GitHub Copilot, might still be GenAIScripts do support inner prompts, but in these used to help write that software. cases the LLM invocation is still statically defined. 3 https://github.com/microsoft/autogen 3 -------- Page Break -------- $`You are an expert Lua programmer and Because LLMs are so flexible, we are just starting to you will write a function that implements a understand the best way to factor the parameterization 5 and specification of tasks in tools like GenAIScript. As Timsortsorting algorithm.` a result, we provide a second, more flexible interface The underlying model may not know much about Lua to provide context to the GenAIScript called a GPSpec programming and/or not be aware of the Timsort algo- (Generative Programming Spec) defined with the ex- rithm. By augmenting the GenAIScript with refer- tension .gpspec.md. This file is a simple markdown ences to context that describes both Lua and Timsort, file that provides more input to the GenAIScript in two the quality of the resulting generation can be greatly ways. Any links present in the gpspec file are accessi- improved. ble in the script via env.links. The textual contents of the gpspec file can also be accessed via env.gpspec. A key abstraction in GenAIScript is the ability to de- fine input sources (that are included in the prompt con- As an example, a GenAIScript can be written to act as text) and allow them to be referred to in the LLM a generic expert code generating program in a specific prompt. language, such as our example of Lua above. In this use case, a GPSpec file could be written that directs Context (env + def) the GenAIScript on the specific code generating task at hand (such as writing an implementation of Tim- In GenAIScript, the env variable forms the root of con- sort). tent from which the LLM prompt context can be gen- Commented [BZ4]: Need a better example/explanation of erated. Using env to define the context allows the using a GPSpec here. Specifying and Verifying Outputs scripts to be portable and easily parameterized. For Commented [BZ5]: We special case output to files by tell- example, env often refers to the directory in which the A key element of robust software is the ability to spec- ing the LLM how to express that kind of output. Should we script is run and env.files refers to all the files in that ify and check the output of a function. Because LLMs mention this in the paper? directory. The command: def(“FILES”, env.files) eas- by their design generate sequences of tokens, layers of ily provides the content of all the files in the directory software are needed to enforce more structure on the to the LLM. generated output. For example, it is often desirable that the LLM output be syntactically well-formed In the example above, we could place two files in the JSON. current directory: 1) an overview of the Lua program- ming language in language.pdf, and 2) a description of Numerous efforts have been made to constrain LLM the Timsort algorithm in sort.pdf. We can then greatly 6 output. GenAIScript supports several mechanisms to improve the LLM performance on the above prompt enforce output specifications on the LLM generations. with the code: First, it provides a mechanism, defSchema, to define a JSON/YAML format for the generated output. Fol- def(“LANG”, env.files, {matches: “language”}) Commented [BZ3]: I need to find the best way to embed lowing the “All You Need Is Types” approach from code into a Word doc. TypeChat, the schema is converted to TypeScript types def(“SORT”, env.files, {matches: “sort”}) before being injected in the LLM prompt. When def- We can now rewrite the prompt above to make the re- Schema is used, GenAIScript will automatically vali- sulting script much more general and more likely to date the resulting generation against the specification generate a correct result: and either attempt repair or fail, depending on what the script specifies. $`You are an expert in the language described in LANG and you will write a function that imple- GenAIScript also supports a more sophisticated con- ments the sorting algorithm described in SORT.` trol over LLM generation via the AI Controller Inter- 7 face system (AICI).Using AICI, the script can define Using the def and env mechanisms in GenAIScript, the much richer constraints on the LLM generations. Ex- above script is entirely parameterize by both program- amples of the kinds of constraints expressible can be ming language and sorting algorithm. GPSpec 5 6 Timsort is a real sorting algorithm, described here: Pydantic validators (Minimize LLM Hallucinations Timsort — the fastest sorting algorithm you’ve never with Pydantic Validators - Pydantic Blog), TypeChat heard of | by Brandon Skerritt | HackerNoon.com | Me- (https://microsoft.github.io/TypeChat/blog/introduc- dium ing-typechat/), etc. 7 AICI | GenAIScript (microsoft.github.io) 4 -------- Page Break -------- 8 found in the Guidance prompt language. For example, 7. Next Steps the user can specify that the LLM output must match an arbitrary regular expression or be an exact substring 8. Conclusion of the input prompt. Ultimately, we believe that the ability to express and enforce rich constraints on LLM generations will be critical in the widespread adoption of AISW. Our goal with GenAIScript is to support the latest developments in such specifications and incorporate the related im- plementation mechanisms (such as AICI) into our lan- guage runtime. 4. GenAIScript Implementation Based on the principles and mechanisms defined above, we provide an overview of how GenAIScript is implemented. In its current form, a GenAIScript is a JavaScript pro- gram that incorporates several enhancements: - Syntax modifications to easily define con- text, prompts, etc. - Targeted LLM system prompts to support communicating information about both ex- pected inputs and outputs to the LLM - Runtime libraries that support LLM interac- tions and parsing LLM outputs. We consider each of these elements in turn. GenAIScript Syntax 5. Case Studies 6. Related Work Examples: - Langchain - Semantic kernel - Autogen 8 guidance-ai/guidance: A guidance language for con- trolling large language models. (github.com) 5 ``` Given the paper in FILE, write a 140 character summary of the paper that makes the paper sound exciting and encourages readers to look at it. ````

👾 system

- tokens: 8 ```markdown You are concise. Answer in markdown. ```

```js system({ title: "Markdown system prompt" }) $`You are concise. Answer in markdown.` ```

👾 system.explanations

- tokens: 9 ```markdown When explaining answers, take a deep breath. ```

```js system({ title: "Explain your answers" }) $`When explaining answers, take a deep breath.` ```

👾 system.files

- tokens: 146 ``````markdown ## Files When generating or updating files you will use the following syntax: File ./file1.ts: ```typescript What goes in ./file1.ts. ``` File ./file1.py: ```python What goes in ./file1.py. ``` File /path_to_file/file2.md: `````markdown What goes in /path_to_file/file2.md. ````` Make sure to use precisely ``` to guard file code sections. Make sure to use precisely ````` to guard file markdown sections. Use full path of filename in code section header. If a file does not have changes, do not regenerate. Do NOT emit line numbers in file. CSV files are inlined as markdown tables. ``````

```js system({ title: "File generation", description: "Teaches the file format supported by GenAIScripts", }) const folder = env.vars["outputFolder"] || "." $`## Files When generating or updating files you will use the following syntax:` def( `File ${folder}/file1.ts`, `What goes in\n${folder}/file1.ts.`, { language: "typescript" } ) def( `File ${folder}/file1.py`, `What goes in\n${folder}/file1.py.`, { language: "python" } ) def( `File /path_to_file/file2.md`, `What goes in\n/path_to_file/file2.md.`, { language: "markdown" } ) $`Make sure to use precisely ${env.fence} to guard file code sections.` $`Make sure to use precisely ${env.markdownFence} to guard file markdown sections.` $`Use full path of filename in code section header.` if (folder !== ".") $`When generating new files, place files in folder "${folder}".` $`If a file does not have changes, do not regenerate.` $`Do NOT emit line numbers in file.` $`CSV files are inlined as markdown tables.` ```

🧠 llm request (2 messages)

- tokens: 4306 - temperature: 0.2 - api type: azure - model: gpt-4 - url: [https://tnrllmproxy.azurewebsites.net/openai/deployments/gpt-4/chat/completions?api-version=2023-09-01-preview](https://tnrllmproxy.azurewebsites.net/openai/deployments/gpt-4/chat/completions?api-version=2023-09-01-preview)

✉️ messages

``````json { "temperature": 0.2, "messages": [ { "role": "system", "content": "---\nYou are concise. Answer in markdown.\n\n---\nWhen explaining answers, take a deep breath.\n\n---\n## Files\n\nWhen generating or updating files you will use the following syntax:\nFile ./file1.ts:\n```typescript\nWhat goes in\n./file1.ts.\n```\n\nFile ./file1.py:\n```python\nWhat goes in\n./file1.py.\n```\n\nFile /path_to_file/file2.md:\n`````markdown\nWhat goes in\n/path_to_file/file2.md.\n`````\n\nMake sure to use precisely ``` to guard file code sections.\nMake sure to use precisely ````` to guard file markdown sections.\nUse full path of filename in code section header.\nIf a file does not have changes, do not regenerate.\nDo NOT emit line numbers in file.\nCSV files are inlined as markdown tables.\n\n" }, { "role": "user", "content": [ { "type": "text", "text": "FILE:\n```pdf file=\"GenAIScript-Introduction-March2024.pdf\"\nGenAIScript: Robust Scripting for Generative AI\n\nPeli de Halleux, Markus Kuppe, Michał Moskal, Madan Musuvathi, Benjamin Zorn\n\nMarch 2024\n\nAbstract our existing computing infrastructure, AISW, with its\nenhanced capabilities, will dominate the development\nScripting languages, like Python and JavaScript, have\nof future software systems. Our goal is to make\nenabled major advances in computing but were created\nGenAIScript the language for scripting AISW.\nprior to the advent of Large Language Models\n(LLMs). This paper introduces GenAIScript, a script- Historically, scripting languages like Perl, JavaScript,\ning language that makes LLMs a first-class part of the and Python, have become widely successful because\nscripting process, allowing users to author, debug, and they provide two key elements: (1) they allow diverse\ndeploy LLM-based scripts that can perform tasks be- users to write small but useful programs, and (2) they\nyond the reach of conventional code. By extending the give users access to new computing capabilities not\npopular Visual Studio Code environment, GenAIS- previously available to programs (e.g., JavaScript gave\ncript simplifies the creation of robust and reusable its users access to the Web).\nscripts that can intelligently process and generate con-\nGiven that context, the goal of GenAIScript is to em-\ntent, including code, documentation, and structured\npower a wide class of potential users to easily leverage\ndata. We explain what GenAIScript is, how it is im-\nthe power of AISW. GenAIScript enables users, in-\nplemented, and illustrate its use in several production\ncluding those without extensive programming exper-\ndeployments.\ntise, to author, debug, and deploy scripts that seam-\nlessly incorporate calls to LLMs. GenAIScript is a Ja-\n1. Introduction\nvaScript-compatible language with a dedicated VS\nCode extension to simplify the scripting process, mak-\n1\nGenAIScript is a JavaScript-compatiblescripting lan-\ning it accessible to a broader audience.\nguage that allows users to write scripts to automate\ncomplex tasks by leveraging LLMs as a first-class\nWith the goal of making GenAIScript the go-to lan-\nscript element.\nguage for writing AISW, we have the following key\nobjectives:\nCommented [BZ1]: @Madan Musuvathi - I added a list\nThe dramatic rise in the capabilities of artificial intel-\nof key objectives and how we address them related to your\nligence has revolutionized the way we interact with\n1. Integration with LLMs: GenAIScript is de-\nquestion. Let me know what you think.\ntechnology, offering unprecedented opportunities to\nsigned to make LLMs a first-class part of the\nautomate and enhance many tasks. Incorporating calls\nscripting process, allowing users to author,\nto LLMs (or more generally any foundation model) at\ndebug, and deploy LLM-based scripts.\nruntime into a software system greatly enhances what\nthat software can do. The disruptive nature of this\n2. Simple Abstractions: GenAIScript provides\ntransition forces us to categorize software into Plain\nabstractions to define the prompt context, re-\nOrdinary Software (POSW), which does not use the\nfer to the context in the prompt itself, and\npower of LLMs at runtime, and the more capable AI\nsupports specification and checking of multi-\nSoftware (AISW) that does leverage LLMs.\nple output formats.\nPOSW represents all software written before the crea-\n3. Ease of Use: GenAIScript targets a wide\ntion of LLMs and includes operating systems, applica-\nclass of potential users, including non-devel-\ntions, scripts, etc. While POSW is a key element of\nopers. The VS Code extension and\n\n1\nCurrently GenAIScripts are JavaScript-compatible\nbut in the future we anticipate bindings for other lan-\nguages including Python.\n\n1\n\n-------- Page Break --------\n\nintegration with existing IDE features con- Because every GenAIScript calls an LLM, metadata is\ntribute to this quality. needed to define the parameters that will be used to\ninvoke the LLM. Those parameters can include the\n4. Robustness: GenAIScript aims to be robust\nmodel, the temperature, the number of tokens to re-\nby leveraging the capabilities of VS Code for\nturn, etc. By default, GenAIScripts use the “gpt-4”\nwriting, executing, debugging, and refining\nmodel and other reasonable defaults, so many of the\nscripts. Additional features like input and out-\nmetadata fields can be omitted.\nput specifications and static orchestration\ngraphs also contribute to robustness. A GenAIScript executes in three parts: 1) set up the\nLLM prompt, 2) execute the LLM, and 3) parse the\n5. Seamless Integration with Plain Old Soft-\nLLM output.\nware (POSW): GenAIScript is designed to\nbe embedded into existing POSW work- Setting up the prompt is done with the “def” command,\nflows, allowing users to integrate AI capabil- which in this case associates the name FILES with the\nities into traditional software systems. contents of env.files. env.files is a variable that refers\nto the parameters (typically one or more files) that the\nIn the remainder of this paper, we describe the user ex-\nscript is invoked on. After this def command, when the\nperience of creating a GenAIScript, define key\nuser refers to FILES in the prompt, the LLM will know\nGenAIScript abstractions, highlight the capabilities of\nthey are referring to the contents of the files in\nthe framework, and describe several case studies based\nenv.files.\non our experiences building and deploying GenAIS-\ncripts in production setting. We discuss related work The text between $`…` in the figure is the prompt that\nand also our thoughts about future additions and fea- the LLM will process. The $`…` instructs the LLM\nwhat to do and refers to the names defined using def\ntures.\nsuch as FILE. This prompt is typically the final part\nof every GenAIScript. The prompt, augmented by any\n2. Introducing GenAIScript\ncontent defined by defs, will be passed to the LLM.\nWhen the LLM returns a result, it will be either be pro-\nGenAIScript was designed to provide a low barrier to\ncessed by GenAIScript, or returned directly.\nentry so that many users, including non-developers,\ncan quickly start using and writing scripts. However,\nThis example, when executed, will shorten the con-\nwe also want to enable power-users to leverage their\ntents of the input file as directed by the prompt. While\nprograming skills to create and deploy more sophisti-\nthe LLM generates a string as output, the GenAIScript\ncated scripts for more challenging scenarios. As a re-\nruntime understands standard output file formats, such\nsult, GeneAIScript is JavaScript compatible where the\nas code files (Python, etc.), markdown files, etc., so\nscripts can contain JavaScript code but less sophisti-\nthat if the prompt instructs the model to produce a file\ncated users can still define powerful and effective\nof a given type, the runtime will parse the resulting\nscripts using the simple abstractions we provide.\ngeneration and create and populate the file.\nIn this example, because the prompt says “shorten the\nfollowing FILE”, the LLM will generate another file\nTo understand GenAIScript, we present a simple com-\nwith the same name. The GenAIScript runtime will\nplete script in Figure 1.\ncreate the new version of the file and then, using the\nexisting refactoring feature that is part of VS Code,\npresent the differences between the original file and\nthe new file in the UI and ask the user to confirm the\n\nFigure 1: A simple GenAIScript\n\nThis example illustrates the key elements of every\nCommented [BZ2]: @Peli de Halleux , @Markus Kuppe ,\nGenAIScript: the metadata, the context, and the\n@Madan Musuvathi I added this description to the white\nprompt. We describe each element in turn.\npaper. Let me know what you think.\n\n2\n\n-------- Page Break --------\n\nedits (shown in Figure 2). research but due to the uncertain nature of the quality\nand robustness of multi-agent interactions, in GenAIS-\ncript we have chosen a different approach toward the\ngoal of creating more robust AISW.\nSpecifically, in GenAIScript, when a script executes\n4\nthere is a single call to the LLM. This approach al-\nlows the user to apply traditional techniques to im-\nprove their script’s software robustness (testing, etc.)\nleveraging many existing tools (debugging, refactor-\ning, etc.) that are part of IDEs like VS Code.\nEvery GenAIScript can be called from the command\nline. As a result, we encourage the integration of our\nFigure 2: Refactoring view of LLM output in GenAIScript\nscripts into existing POSW automation processes. We\nfrom script in Figure 1.\nbelieve that focusing the AISW on the specific hard\ntasks that POSW can’t do, and leveraging the enor-\n3. GenAIScript Abstractions\nmous robust POSW infrastructure, is the best way to\navoid known robustness challenges in building AISW.\nThe design of GenAIScript follows several principles:\nAs an aside, if a user writes a GenAIScript to generate\n- LLM at the bottom. By this we mean that\na program as output, say in Python, it is not automati-\nevery GenAIScript will use an LLM to\ncally executed. If the user wants to use GenAIScript\nachieve its goal. We firmly believe that if the\nto help write software by generating it automatically,\nuser wants to accomplish something that is\nthey still have to edit/execute/debug that software in a\neasily accomplished with POSW, they should\ntraditional manner.\n2\nuse that instead. Use GenAIScript for prob-\nlems that POSW cannot solve.\nParameterization\n- Control AI Behavior with a Static Orches-\ntration Graph. We do not permit the LLM\nFor scripts written in POSW, the parameters provide a\nin GenAIScripts to determine other AI\nway to generalize and customize the application of the\nscripts/agents to call and/or call them. In-\nscript. For example, they might specify the name of\nstead, a user can determine statically exactly\nan input file so that the script can be applied to many\nwhat LLM will be called and with prompt\ndifferent files. In general, for POSW, the input file\ncontext.\ndoes not affect how well the script itself executes or\n- Parameterizing context. We provide mech-\nwhat it does.\nanisms to bring external context into\nIn GenAIScript, the input provided can have enormous\nGenAIScripts as well as provide mechanisms\nimpact on what the script does and how well it does it.\nto structure that input to make it more com-\nThis happens because inputs to the script are:\nprehensible to the LLM.\n- Specifying and verifying outputs. GenAIS-\n- Typically text files provided for Retrieval\ncript supports numerous ways to specify and\nAugmented Generation (RAG)\ncheck output requirements for the AI-gener-\n- Almost always passed to the underlying LLM\nated content to ensure it conforms to the user\nto help it be more effective.\nspecifications.\nFor example, the prompt in a script might say:\nWith these principles in mind, we discuss the key\nGenAIScript abstractions.\nStatic Orchestration Graph\nApproaches to allowing a collection of AI agents to be\ndefined and integrated have been proposed (for exam-\n3\nple, autogen). This is an exciting area of current\n\n2 4\nNote that AI, such as GitHub Copilot, might still be GenAIScripts do support inner prompts, but in these\nused to help write that software. cases the LLM invocation is still statically defined.\n3\nhttps://github.com/microsoft/autogen\n\n3\n\n-------- Page Break --------\n\n$`You are an expert Lua programmer and Because LLMs are so flexible, we are just starting to\nyou will write a function that implements a understand the best way to factor the parameterization\n5\nand specification of tasks in tools like GenAIScript. As\nTimsortsorting algorithm.`\na result, we provide a second, more flexible interface\nThe underlying model may not know much about Lua\nto provide context to the GenAIScript called a GPSpec\nprogramming and/or not be aware of the Timsort algo-\n(Generative Programming Spec) defined with the ex-\nrithm. By augmenting the GenAIScript with refer-\ntension .gpspec.md. This file is a simple markdown\nences to context that describes both Lua and Timsort,\nfile that provides more input to the GenAIScript in two\nthe quality of the resulting generation can be greatly\nways. Any links present in the gpspec file are accessi-\nimproved.\nble in the script via env.links. The textual contents of\nthe gpspec file can also be accessed via env.gpspec.\nA key abstraction in GenAIScript is the ability to de-\nfine input sources (that are included in the prompt con-\nAs an example, a GenAIScript can be written to act as\ntext) and allow them to be referred to in the LLM\na generic expert code generating program in a specific\nprompt.\nlanguage, such as our example of Lua above. In this\nuse case, a GPSpec file could be written that directs\nContext (env + def)\nthe GenAIScript on the specific code generating task\nat hand (such as writing an implementation of Tim-\nIn GenAIScript, the env variable forms the root of con-\nsort).\ntent from which the LLM prompt context can be gen-\nCommented [BZ4]: Need a better example/explanation of\nerated. Using env to define the context allows the\nusing a GPSpec here.\nSpecifying and Verifying Outputs\nscripts to be portable and easily parameterized. For\nCommented [BZ5]: We special case output to files by tell-\nexample, env often refers to the directory in which the\nA key element of robust software is the ability to spec-\ning the LLM how to express that kind of output. Should we\nscript is run and env.files refers to all the files in that\nify and check the output of a function. Because LLMs\nmention this in the paper?\ndirectory. The command: def(“FILES”, env.files) eas-\nby their design generate sequences of tokens, layers of\nily provides the content of all the files in the directory\nsoftware are needed to enforce more structure on the\nto the LLM.\ngenerated output. For example, it is often desirable\nthat the LLM output be syntactically well-formed\nIn the example above, we could place two files in the\nJSON.\ncurrent directory: 1) an overview of the Lua program-\nming language in language.pdf, and 2) a description of\nNumerous efforts have been made to constrain LLM\nthe Timsort algorithm in sort.pdf. We can then greatly 6\noutput. GenAIScript supports several mechanisms to\nimprove the LLM performance on the above prompt\nenforce output specifications on the LLM generations.\nwith the code:\nFirst, it provides a mechanism, defSchema, to define a\nJSON/YAML format for the generated output. Fol-\ndef(“LANG”, env.files, {matches: “language”})\nCommented [BZ3]: I need to find the best way to embed\nlowing the “All You Need Is Types” approach from\ncode into a Word doc.\nTypeChat, the schema is converted to TypeScript types\ndef(“SORT”, env.files, {matches: “sort”})\nbefore being injected in the LLM prompt. When def-\nWe can now rewrite the prompt above to make the re-\nSchema is used, GenAIScript will automatically vali-\nsulting script much more general and more likely to\ndate the resulting generation against the specification\ngenerate a correct result:\nand either attempt repair or fail, depending on what the\nscript specifies.\n$`You are an expert in the language described in\nLANG and you will write a function that imple-\nGenAIScript also supports a more sophisticated con-\nments the sorting algorithm described in SORT.`\ntrol over LLM generation via the AI Controller Inter-\n7\nface system (AICI).Using AICI, the script can define\nUsing the def and env mechanisms in GenAIScript, the\nmuch richer constraints on the LLM generations. Ex-\nabove script is entirely parameterize by both program-\namples of the kinds of constraints expressible can be\nming language and sorting algorithm.\nGPSpec\n\n5 6\nTimsort is a real sorting algorithm, described here: Pydantic validators (Minimize LLM Hallucinations\nTimsort — the fastest sorting algorithm you’ve never with Pydantic Validators - Pydantic Blog), TypeChat\nheard of | by Brandon Skerritt | HackerNoon.com | Me- (https://microsoft.github.io/TypeChat/blog/introduc-\ndium ing-typechat/), etc.\n7\nAICI | GenAIScript (microsoft.github.io)\n\n4\n\n-------- Page Break --------\n\n8\nfound in the Guidance prompt language. For example,\n7. Next Steps\nthe user can specify that the LLM output must match\nan arbitrary regular expression or be an exact substring\n8. Conclusion\nof the input prompt.\nUltimately, we believe that the ability to express and\nenforce rich constraints on LLM generations will be\ncritical in the widespread adoption of AISW. Our goal\nwith GenAIScript is to support the latest developments\nin such specifications and incorporate the related im-\nplementation mechanisms (such as AICI) into our lan-\nguage runtime.\n\n4. GenAIScript Implementation\nBased on the principles and mechanisms defined\nabove, we provide an overview of how GenAIScript is\nimplemented.\nIn its current form, a GenAIScript is a JavaScript pro-\ngram that incorporates several enhancements:\n- Syntax modifications to easily define con-\ntext, prompts, etc.\n- Targeted LLM system prompts to support\ncommunicating information about both ex-\npected inputs and outputs to the LLM\n- Runtime libraries that support LLM interac-\ntions and parsing LLM outputs.\nWe consider each of these elements in turn.\nGenAIScript Syntax\n\n5. Case Studies\n\n6. Related Work\nExamples:\n- Langchain\n- Semantic kernel\n- Autogen\n\n8\nguidance-ai/guidance: A guidance language for con-\ntrolling large language models. (github.com)\n\n5\n```\n\nGiven the paper in FILE, write a 140 character summary of the paper \nthat makes the paper sound exciting and encourages readers to look at it.\n\n" } ] } ], "stream": true } ``````

- response: 200 OK

📩 llm response

output

````markdown ```markdown Discover GenAIScript: a new scripting language integrating AI models to automate complex tasks, revolutionizing code generation and software dev! ``` ````


Extension version: 1.19.2
VS Code version: Code 1.88.1 (e170252f762678dec6ca2cc69aba1570769a5d39, 2024-04-10T17:41:02.734Z)
OS version: Windows_NT x64 10.0.22631
Modes:

<details>
<summary>System Info</summary>

|Item|Value|
|---|---|
|CPUs|Intel(R) Xeon(R) Silver 4108 CPU @ 1.80GHz (32 x 1796)|
|GPU Status|2d_canvas: enabled<br>canvas_oop_rasterization: enabled_on<br>direct_rendering_display_compositor: disabled_off_ok<br>gpu_compositing: enabled<br>multiple_raster_threads: enabled_on<br>opengl: enabled_on<br>rasterization: enabled<br>raw_draw: disabled_off_ok<br>skia_graphite: disabled_off<br>video_decode: enabled<br>video_encode: enabled<br>vulkan: disabled_off<br>webgl: enabled<br>webgl2: enabled<br>webgpu: enabled|
|Load (avg)|undefined|
|Memory (System)|63.71GB (43.36GB free)|
|Process Argv|--crash-reporter-id 2058db1e-f1e8-433e-bf38-476163c306a2|
|Screen Reader|no|
|VM|0%|
</details><details>
<summary>A/B Experiments</summary>

vsliv368cf:30146710 vspor879:30202332 vspor708:30202333 vspor363:30204092 vscod805:30301674 binariesv615:30325510 vsaa593cf:30376535 py29gd2263:30899288 c4g48928:30535728 azure-dev_surveyone:30548225 a9j8j154:30646983 962ge761:30959799 pythongtdpath:30769146 welcomedialogc:30910334 pythonidxpt:30866567 pythonnoceb:30805159 asynctok:30898717 pythontestfixt:30902429 pythonregdiag2:30936856 pyreplss1:30897532 pythonmypyd1:30879173 pythoncet0:30885854 h48ei257:31000450 pythontbext0:30879054 accentitlementst:30995554 dsvsc016:30899300 dsvsc017:30899301 dsvsc018:30899302 cppperfnew:31000557 d34g3935:30971562 fegfb526:30981948 bg6jg535:30979843 ccp2r3:30993541 dsvsc020:30976470 pythonait:31006305 gee8j676:31009558 dsvsc021:30996838 gd77d436:30999572 jg8ic977:31013176



</details>

<!-- generated by issue reporter -->

microsoft / genaiscript