📢 ChatGPT challenge: try using AI tools in the iP/tP

[damithc]

Tools like ChatGPT and GitHub copilot are likely to have a significant impact on how software engineers work. Although those tools are still in early stages, it is best for us to be at the cutting edge of what they can do. To that end, we would like to encourage you guys to try the following optional explorations:

a) Use AI on the iP: Try to use such AI tools to enhance the iP further. For example, to add more features to the iP, improve the code quality, write more tests, etc. You can push the enhanced version of the code to your fork, after tagging it AiAssisted. This work will not be considered in iP grading; but after some prelim work, if you are interested to do more in that direction, you can consider earning CS2103R credit for such work (see option C here).

b) Use AI in the tP: You can try to use those tools in the tP as well. If it turns out that those tools can save you time without compromising your learning, it is fine to use them in the tP. In addition, learning what those tools can do for you has value too.

In both cases above,

i) do share your interesting experiences (positive as well as negative) in this thread so that others can benefit from them too (plus you get credit for sharing useful things in the forum).

ii) keep in mind that you need to declare non-trivial tP code segments generated by such tools as 'reused'.

• How to mark code segments as 'reused'? See RepoSense instructions here.
• Our caveat on using AI tools is here.

[wxxedu]

I would like to share a bit about my story with Copilot. My friend joined Copilot beta before me, and once we were doing some online coding sessions together, I realized that he wasn't using it. I asked him why, and he said that he found the code generated to be not so reliable. I don't use Copilot now also. After the release of ChatGPT though, I did use it quite frequently, especially for checking new APIs.

I have several issues with Copilot that prevented me from using it:

1. User experience issues: the copilot would disable the snippets/autocomplete features of the IDE, despite sometimes I would like to see these things. Also, sometimes I would like to press the tab key, and this in Copilot would mean using the suggestion provided, which often turns out not to be what I wanted.
2. Bad code: I often find myself, after pressing many tabs, ending up with a bunch of code that doesn't really make sense. I felt a reduction in the quality of the code that I wrote. While when Copilot works, it works quite nicely, there are so many cases when it doesn't work. Also, writing the code has never been the part that takes a longer time, but rather finding the bug and making sure that the code is correct. Further, Copilot does not seem to have a big picture ahead of it. The code that it produces lacks a "taste" (I don't know what better word to describe it).
3. Distraction: there are a lot of times when I started writing code, I would see that Copilot suggesting me a long block of code. Then, I will need to go and read through what it suggested, which distracts me from my own train of thought.

I use ChatGPT as a "search engine" that helps me check APIs and gives me concrete examples of how certain APIs are used. It reduces the distraction of Github Copilot, and it is much faster than searching via Google. There are also many occasions when the APIs are poorly documented, and ChatGPT helps greatly with that. Further, because I generally don't use the code generated by ChatGPT, I feel more control over the code that's written.

I may not hold the popular opinion, but I don't believe, at least at the current stage, that AI can write good code. Perhaps they can write OK code, but I don't think that the code written by AI would be production ready in any sense.

[wxxedu]

It's more sort of like that Copilot/ChatGPT can get an algorithm implemented / an API correctly invoked, but they lack the structural knowledge of the codebase, and they don't know how to structure your code such that it is nice and clean.

[BoAi01]

they lack the structural knowledge of the codebase

I think this is spot on. LLMs (Large-Langauge Models) do not have the context about the high-level system architecture, thus it is only capable of producing short snippets (relatively compared to SE projects). Manual efforts are needed to adapt LLMs' outputs for the purpose of specific projects at hand. It's unlikely that we can get LLMs to generate code at scale.

I use ChatGPT as a "search engine" that helps me check APIs

I use it in exactly the same way. It sometimes works better than StackOverflow, as it only displays the most likely implementation of a certain functionality (in contrast, Google has many answers but many of them are just noise). I turn to GPT first if I think the function I want to implement is common and routine.

[wxxedu]

Just to be a bit more specific, the issue that I found with Copilot is that it tends to cause a lot of code duplication. When two chunks of code have very similar logic, it would just fill out the second part, without considering extracting the shared parts. Another problem that I found with Copilot is that it tends to create unnecessary features.

[damithc]

Thanks @wxxedu @BoAi01 for sharing your observations so far.

I've been using copilot for my coding (Python) for a few weeks now. There are times it gets in the way, but at times it saves me some typing, giving me possibly about 5% productivity gain. It might increase as the model improves. But every time I need to look at the suggested code closely and decide if it's the correct way to do it, which is the hard part of coding. The time saved on typing is only a minor gain in comparison.

[salty-flower]

To add on, these days Cursor looks promising too. Free alternative to Copilot, huh.