humphd
commented
1 year ago Related, it's hilarious how bad GPT-3.5 can be at doing a summary. Here's a GPT-authored summary about ChatCraft features, which it summarizes as "benefits of using a password manager". I'd say we don't have to aim for "AI perfect" since it also doesn't exist :)
We have a very basic summarizer in https://github.com/tarasglek/chatcraft.org/blob/main/src/lib/summarize.ts. It's meant to be small (no deps, runs in browser) and "good enough." We use it to generate summaries to display in the UI so the user doesn't have to a) pay or wait for GPT to write them all; b) enter something manually. It's working well, but...
It needs to be improved for some Markdown cases, like numbered lists, where it chops off the text after the period in
1. this text gets lost...:I didn't spend too much time on this, since I just needed something that worked. Help to improve it would be great.
I generally want to use the first 50-200 characters of a summary in the UI (50 when showing in the window title, 200 everywhere else). So the goal would be: produce a 200'ish character summary that is somewhat Markdown aware, but without pulling in all kinds of NLP code that's going to bloat the app if it can be avoided.
If you're looking to contribute to ChatCraft, this would be a nice isolated place to start. I'd recommend having some chats with ChatCraft.org about the problem and show it the existing code; bonus points if you can fix ChatCraft with ChatCraft.