[BUG] markdown: bulletpoints render without line breaks?

[mindplay-dk]

Description

I'm trying out Big-AGI with Claude 3.5 Sonnet, and it seems the UI can't handle bulletpoints in markdown.

Inspecting the HTML in the browser, it's just a <p> tag and a bunch of bullet point characters:

I tried exporting as markdown, and here it's unicode bulletpoints as well.

If I run the same query in Claude's UI, the HTML is proper <li> elements - however, if I press the "copy" button on the chat response, the markup is those same unicode bulletpoint characters instead.

This might suggest that's actually how Claude responds now? Which seems like a really odd choice on their part.

I obviously can't fault you (or whatever library you're using) for not handling something that isn't proper markdown - but it sounds really odd. Has anthropic decided to go it's own way and just invent some weird new markdown syntax here? Just seems like an unnecessary roadblock for anything that relies on any of the most common markdown libraries.

But I suppose, if this is how Anthropic does it now, maybe that's something clients will need to handle? 🤷‍♂️

Very odd. 🤨

Device and browser

Chrome on Win 11

Screenshots and more

Willingness to Contribute

• [ ] 🙋‍♂️ Yes, I would like to contribute a fix.

[enricoros]

@mindplay-dk very interesting. So in this case it's detection of the unicode bullets, and rendering them as lists?

What's your advice to enable a good rendering, and also a good copy of message and copy of selected experience?

[mindplay-dk]

Good questions.

Yeah, I don't guess there's any way to get the attention of a big corporation like Anthropic and make them aware of this - and it's probably too late anyhow, since they've most likely trained their LLM to use this peculiar non-standard markdown syntax.

So it's a matter of supporting that character on equal footing with * and - at the parser level, I suppose? The syntax rules in terms of white space appears to be otherwise the same, so.

I can see you're using react-markdown, which uses remark, which looks quite modular - but unfortunately only appears to provide support for exactly two bullet types via configuration: bullet and bulletOther.

https://github.com/remarkjs/remark/tree/main/packages/remark-stringify#options

Odd choice. 🤔

I mean, I'm sure there's a good reason why the library treats these so differently - without looking into the code, I'd guess, probably something to do with syntax priorities in the parser.

But we don't want to replace either the * or the - when parsing Anthropic's output, I don't think? Without a doubt the training set would have included markdown that uses the * or - syntax, so it's hard to think it's guaranteed to never use that syntax? I supposed only someone from Anthropic would know.

So it doesn't look like we can solve this with simple configuration.

You might could solve it with a plugin:

https://github.com/syntax-tree/mdast-util-from-markdown

But honestly, the simplest solution here is probably just regex replace that character with a * - if you can figure out some way to do that specifically for your Anthropic API adapter only?

text.replace(/^[ \t]*•/gm, '*')

It's a bit rugged, but it'll work, I think? I don't know what this character would be commonly used for anything else. It could be - I mean, you could prompt it specifically to use that character for something... but it seems unlikely, right?

Ideally, I'd really like to get a word from Anthropic on this - because I strongly suspect they did not somehow manage to erase standard markdown syntax from Claude's memory.

What I suspect is that someone wanted these unicode bulletpoints in the text output for non technical users, and for some reason (rookie mistake) decided to implement this at the API level, rather than at the UI level of their chat client.

[mindplay-dk]

FYI, I've reached out to Anthropic support to make them aware of this issue.

[mindplay-dk]

I poked around to see if I could fix this - I just changed the docker-compose command from next start to next dev, but:

big-agi-1  |  ⚠ You are using a non-standard "NODE_ENV" value in your environment. This creates inconsistencies in the project and is strongly advised against. Read more: https://nextjs.org/docs/messages/non-standard-node-env
big-agi-1  |    ▲ Next.js 14.1.4
big-agi-1  |    - Local:        http://localhost:3000
big-agi-1  | 
big-agi-1  | Error: > Couldn't find any `pages` or `app` directory. Please create one under the project root
big-agi-1  |     at findPagesDir (/app/node_modules/next/dist/lib/find-pages-dir.js:42:15)
big-agi-1  |     at initialize (/app/node_modules/next/dist/server/lib/router-server.js:75:69)
big-agi-1  |     at async Server.<anonymous> (/app/node_modules/next/dist/server/lib/start-server.js:247:36)
big-agi-1  | 
big-agi-1 exited with code 0

Looks like it's set up to copy the production files only.

It might be nice to get that fixed, and maybe get a devcontainer.json file in there, so contributors can get to work directly? There are so many versions of node, and system dependencies, and always so many problems getting projects configured to run and debug locally, I just don't have the patience anymore. 😄

Might make sense to do it "once and for all", so every contributor can enjoy that?

[mindplay-dk]

This is soooo interesting.

If I prompt Claude with a simple query like:

Create a bulletpoint list of UX design brainstorming techniques - just a name and a single sentence describing each. Respond using standard markdown.

It responds with the weird bulletpoints:

Here's a concise list of UX design brainstorming techniques with brief descriptions:

• Mind mapping: Visual diagram connecting ideas around a central concept.

• 6-3-5 Brainwriting: Six people write three ideas in five minutes, then pass to the next person.

• Crazy 8s: Sketch eight distinct ideas in eight minutes.

...

However, if I specifically ask for "standard markdown":

Create a bulletpoint list of UX design brainstorming techniques - just a name and a single sentence describing each. Respond using standard markdown.

Then I get:

Here's a bulletpoint list of UX design brainstorming techniques using standard markdown:

- **Mind Mapping**: Create a visual diagram to explore and organize ideas around a central concept.

- **Six Thinking Hats**: Approach problems from six different perspectives to generate diverse ideas.

- **Crazy Eights**: Rapidly sketch eight distinct ideas in eight minutes to encourage quick, creative thinking.

...

I thought maybe the word "bulletpoint" was throwing it off, but even if I remove that and ask for an "unordered list", it prefers those wonky bulletpoints. So it's not a bug in their API or something - they must have intentionally finetuned it to output non-standard markdown syntax for some reason.

I went and look at their recently-leaked system prompt, and it's not like they're prompting it specifically to respond with non-standard markdown, so the only explanation is fine-tuning.

This soooo bizarre. 😄

[enricoros]

Thai is very interesting. Maybe Claude is actually prompted less heavy handedly compared to GPT-4 and their variants, possibly due to Anthropic preferring XML structures and generalization rather than baking in correct markdown as the response format. Maybe the prompt should have some hints of using correct markdown ("... Respond in valid markdown.", or "Markdown: standard.")

[mindplay-dk]

Maybe the prompt should have some hints of using correct markdown ("... Respond in valid markdown.", or "Markdown: standard.")

I was thinking the same thing, and was actually hoping there would be a way to customize the system prompt as a temporary workaround - but it doesn't look like this app has that yet?

There's "personas", which also appear to generate system prompts, but it looks like those are tailored to generate persona type system prompts based on examples, e.g. getting it to write like I do?

Is a persona essentially a system prompt? I couldn't find any way to just "edit" a persona, rather than generating one - if there were some way to edit rather than generate, that might work?

What exactly happens if you issue a system prompt to Claude? does it override it's own built-in in system prompt, that one that's used in Claude's own chat interface? or does it just append your instructions to the built-in ones?

[enricoros]

You can customize the system prompt on a per-conversation basis, just toggle the Menu > Show System Prompt and edit it.

As I'm working on 2.0 (which you're going to love, I'm sure, the multi modality is working incredibly across all vendors) I'll make the whole prompt editing experience, via Personas, better.

The system prompt is represented as the first message in a conversation, and represents exactly what gets sent to Claude as the system instruction. Without it the model follows its own implicit alignment.

Claude follows the system prompt well, and you'll see that the one we use in Big-AGI already has a couple of tricks inside to be more useful. See the picture below, you can edit the default prompt and add:

Markdown format: standard

or something similar. This will go into the system prompt, but unfortunately needs to be copied/pasted on new chats (will fix that).

[mindplay-dk]

You can customize the system prompt on a per-conversation basis, just toggle the Menu > Show System Prompt and edit it.

Oh, I knew that - since it still needs to be done per-conversation, there's really no difference from just prompting it when starting a new conversation though. Works just as well.

It makes me think though (and this might be a nice feature either way) it might be nice to have a simple text input for "global" user-specific instructions, which would be added to the system prompt for all personas - a pretty simple and potentially very useful little feature you might consider. 🙂

[enricoros]

Thanks, added to the list. It's actually easy and useful and quick to do.

I'm still undecided whether to work and the personas system overhaul for Big-AGI 2.0 or release it sooner and then add Personas 2.0. (it's 18,000 lines of code changed already).

Any other compelling features you'd need?

[mindplay-dk]

Any other compelling features you'd need?

It sounds like you already have more than enough on your plate! 😅

I'm still undecided whether to work and the personas system overhaul for Big-AGI 2.0 or release it sooner [...]

Personally, if I can ship a useful feature, and if the next "level up" is more than 2x the work, I'd prefer to get the first version of the feature out sooner - that way, you learn how important or popular the feature is, which can help you prioritize between "2.0" versions of different features. 🙂

Have you thought about containerizing your dev environment? I would have liked to help, but I would want to spend a few hours actually working on a feature, rather than a few hours getting the project set up. 😅

(either way, it sounds like there are so many changes in the pipeline right now, it's probably not the best time for a contributor to onboard - I'm sure there will be lots of breaking changes in 18K lines? and thus merge conflicts.)