sagemathinc / cocalc

CoCalc: Collaborative Calculation in the Cloud
https://CoCalc.com
Other
1.17k stars 216 forks source link

provide a way for instructors to customize the AI prompts that their students see #7803

Open williamstein opened 2 months ago

williamstein commented 2 months ago

I allow students to use the "help me fix this" feature. It's useful especially at the beginning of the course when students are still struggling with syntax issues. However, the features is often too helpful, simply providing drop-in replacements for their code. I wonder if there's a way we could customize whatever prompt is going to the AI agent and ask it to not actually write any code, but just explain in words what's going on. Maybe in the course settings, we can see what the default prompt text is for the AI buttons and potentially modify it for our course. -- Bill Wolf

Customizing prompts makes a lot of sense, but there are also a LOT of different scopes at which to do so:

There's also a lot of questions about which prompts to customize, what the UI is like for doing so, how we explain which is which, etc.

wmwolf commented 2 months ago

Right now, I'm interested in customizing the prompt only for students enrolled in a course (so per-project, I suppose) and only for the case when the student presses the "explain this" or "help me fix this" buttons. These settings could then live in the course configuration file for the instructor to peruse.

I understand that students can always get around these by using third-party services or even just opening up their own project, but having the provided button often just spitting out complete solutions is not conducive to learning coding for the first time... but being able to have students figure out what "int not callable" means right when they encounter it is extremely useful, so I'm hopeful that customizing the prompts in these narrow circumstances could make those buttons more useful for teachers.

williamstein commented 2 months ago

These settings could then live in the course configuration file for the instructor to peruse.

It would have to get copied from the course configuration to somewhere else (maybe a file in the project?), since the student obviously doesn't have access to the course file (as that would be a security issue).

Given your original question, I wonder if just replacing the "Help me fix" button by an "Explain this" button is what you really want?

wmwolf commented 2 months ago

That would also probably solve the problem. A different, more neutered button that can explain errors but does not produce code in its output that students could just drop in blindly.

williamstein commented 2 months ago

This could be next to the checkbox to disable ai completely. Maybe it would be called "neuter ai" or something...