An Example of Few Shot Prompting to generate docstrings with GPT4

[bmwoodruff]

While experimenting with prompt engineering, to reduce costs I'll be using GPT4 to design things before I use Nebari.

As there are already lots of functions in NumPy with docstrings, we can use few-shot prompting to train AI how to provide appropriate NumPy docstrings. We can then generate docstrings for functions that already have them, so that we can compare the generated output with what's currently in the codebase. This is an experiment with the ideas provided at https://github.com/possee-org/possee-resources/issues/1. We could then use this to produce docstrings for functions that do not currently have them.

First, we need a tool that will extract a function from a file and separate the function from its docstring. Then we iterate over this function to generate a few-shot prompt which we can copy/paste into GPT-4 (or algorithmically provide to Llama3 using the API in Nebari). I've created a Jupyter notebook to do this.

• The notebook is located in this repo at https://github.com/possee-org/genai-numpy/blob/main/docstrings/few-shot-docstring-prompt-creator.ipynb
• The file I'm extracting functions and docstrings from is _linalg.py. You'll need this file to be in the same directory.
• I used the docstrings from tensorsolve and transpose to generate docstrings for solve.
• The notebook has links to four GPT4 responses.

The generated docstrings were decent, included examples, and had the proper formatting. Things look promising.

[bmwoodruff]

Here's some screenshots of the results. The third option had an extra newline at the bottom of the prompt, which might be the reason the function code was included as well.