MDN can now automatically lie to people seeking technical information

[eevee]

Summary

MDN's new "ai explain" button on code blocks generates human-like text that may be correct by happenstance, or may contain convincing falsehoods. this is a strange decision for a technical reference.

URL

https://developer.mozilla.org/en-US/docs/Web/CSS/grid

Reproduction steps

as soon as i heard about this, i visited the first MDN article in my address bar history (for the grid property), hit "ai explain" on the first code block encountered (the syntax summary), and received the following information:

grid: "a" 100px "b" 1fr;: This value sets the grid template to have two rows and two columns. The first row has a height of 100 pixels and the second row has a height of 1 fraction unit (1fr). The columns are named "a" and "b".

which is deeply but subtly incorrect — this creates only one column (more would require a slash), and the quoted strings are names of areas, not columns. but it's believable, and it's interwoven with explanations of other property values that are correct. this is especially bad since grid is a complex property with a complex shorthand syntax — exactly the sort of thing someone might want to hit an "explain" button on.

the generated text appears to be unreviewed, unreliable, unaccountable, and even unable to be corrected. at least if the text were baked into a repository, it could be subject to human oversight and pull requests, but as best i can tell it's just in a cache somewhere? it seems like this feature was conceived, developed, and deployed without even considering that an LLM might generate convincing gibberish, even though that's precisely what they're designed to do.

and far from disclaiming that the responses might be confidently wrong, you have called it a "trusted companion". i don't understand this.

Expected behavior

i would like MDN to contain correct information

Actual behavior

MDN has generated a convincing-sounding lie and there is no apparent process for correcting it

Device

Desktop

Browser

Firefox

Browser version

Stable

Operating system

Linux

Screenshot

No response

Anything else?

No response

Validations

• [X] I have read the Community Participation Guidelines.
• [X] I have verified that there isn't already an issue that reports the same bug to avoid creating a duplicate.
• [X] I have checked that this is a concrete bug. For Q&A open a GitHub Discussion.

[ericflo]

This feature is providing incorrect information. How is it a good feature?

When you ask a friend to explain something to you, you don't expect them to get it 100% right, but them explaining it in a different way can help spark understanding. But you do need to know your friend isn't authoritative.

Look, I get it - my opinion isn't shared widely here. But I wanted to share my point of view because I didn't see it expressed by anyone, and I truly think this could be a great feature.

[MrLightningBolt]

This feature is providing incorrect information. How is it a good feature?

When you ask a friend to explain something to you, you don't expect them to get it 100% right, but them explaining it in a different way can help spark understanding. But you do need to know your friend isn't authoritative.

Look, I get it - my opinion isn't shared widely here. But I wanted to share my point of view because I didn't see it expressed by anyone, and I truly think this could be a great feature.

It's a bad opinion. MDN is not my friend, it's a documentation repository.

Beyond that, if my friend sat there confidently lying to me, did not accept updated information and thought that occasionally reminding me that they were irretrievably full of shit made it okay, they would not be my friend for very long.

[alexisvl]

But I wanted to share my point of view because I didn't see it expressed by anyone

I wonder why 🤔

[Sorixelle]

you do need to know your friend isn't authoritative.

Isn't the whole point of MDN to be an explicitly authoritative source of factual documentation for the Web? If tools are being brought in that are presenting users with factually incorrect information, then that's a failure of MDN to achieve it's stated goals. Sure I don't expect a friend to be 100% correct all the time, but I'm also not expecting my friend to be an authoritative figure on web development, unlike MDN.

[jamy015]

I was disappointed to see this on HN today. Simply put, this is the destruction of MDN's hard-earned reputation as a source of accurate, factual docs.

I've written to Mozilla Donor Help to express my disappointment and request a refund for my most recent donation. Mozilla offers refunds up to 15 days after donating, so I would encourage other donors to do the same until this is fixed: https://donate.mozilla.org/en-US/help/

[hlieberman]

I have opened #9209 to revert the merge. Unfortunately, since ChatGPT wrote the PR description, it didn't tag this ticket appropriately, so I am doing it by hand.

[cheshrkat]

This feature is quite a mis-step. First and foremost it undermines the entire point of MDN - providing a reliable and accurate reference. I point learners to MDN specifically because (until now) I didn't have to worry about them getting rubbish information. I don't need MDN to be "clever" I need it to be accurate.

But even if you do believe AI/LLMs are great for learning:

1. MDN's explain button is simply not necessary - if a learner wants an LLM explanation, they can already copy the code into their system of choice and ask for it.
2. The feature doesn't support LLM usage patterns like using follow-up prompts to explore a topic, or ask for corrections.

Basically this feature seems like it would annoy everyone, regardless of their opinion of AI/LLMs. It's a mis-step, and I hope MDN remove it quickly.

[Leo40Git]

I have created a PR to remove this feature, as seen above.

[OnkarRuikar]

Just pointing out a feature we might have missed:

If you select a line of text it explains that only.

[eevee]

If you select a line of text it explains that only.

doesn't that clip to a 0×0 box, not 1×1?

[sideshowbarker]

Just send a pull request reverting it.

Merging a pull request for this on its own doesn't fix it. It needs to also be deployed, and only some Mozilla people have the admin access to deploy.

How come something like this can get added without any review, background information, or informing the Steering Committee

Good question. I aim for the answer going forward to be that it can't, ever again. We can't have them doing anything like this again.

but you need to escalate, discuss, and justify yourself to remove it back?

I wasn't implying that I planned to ask for a discussion about it, or that I felt the need to justify reverting it. This issue on its own should be enough to make it clear that it should be reverted right away. I meant "escalate" more in terms making sure the engineers actually revert and re-deploy right away, as soon as they get online again and are aware.

[Tarmil]

Just pointing out a feature we might have missed:

If you select a line of text it explains that only.

Looks like we need documentation for how AI Explain works. I know, let's ask AI! 🤡

[Zarthus]

Looks like a commit was made by the original author to pause it for now: https://github.com/mdn/yari/commit/1bf285612ca2a2741795916fbe7fd2549e6b0013

[Sorixelle]

Glad it's partially reverted for now, but... commits directly to main, without so much as a PR, or even a message in this issue we have for tracking this? Sounds like Mozilla has some serious accountability concerns to address.

[SudoCat]

Pausing due to "controversy" is hardly enough, and frankly belittles the issue and the community. This should be removed due to being useless and potentially harmful. I cannot believe this feature was even greenlit and shipped.

I fear this is just another step in Mozilla's death spiral.

[IBBoard]

When you ask a friend to explain something to you, you don't expect them to get it 100% right, but them explaining it in a different way can help spark understanding. But you do need to know your friend isn't authoritative.

@ericflo

1) I wouldn't generally treat my friends as authoritative. But documentation should be. 2) Even if they know loads, a friend is likely to couch their uncertainty in "I think it's like this" rather than saying everything with absolute confidence 3) A good friend would understand their limits and either refuse to explain until they explored it again or they'd give their current understanding, check and update you. LLMs don't know that they don't know stuff. 4) I wouldn't generally treat my friends as authoritative. But documentation should be. 5) A good friend would understand their limits and either refuse to explain until they explored it again or they'd give their current understanding, check and update you. LLMs don't know that they don't know stuff.

(Now I realise that technically speaking some of those are the same flaw but I thought that they were such a big ones that it was worth mentioning twice)

[sideshowbarker]

Update: https://github.com/mdn/yari/actions/runs/5429890019 is the GitHub Action for deploying the revert to production. That action typically takes about 45 minutes or less, but can sometimes take as much as 60 minutes. As I write this, it’s currently 35 minutes in, so it should complete in about 10-15 minutes — so, by 08:30 UTC.

[DavidLibeau]

It also says that Array.prototype.with() does not exist…

[robinwhittleton]

A little bit sad that it’s “paused for now” rather than completely reverted, but it resolves the immediate issue. Thanks @sideshowbarker.

[sideshowbarker]

Update: https://github.com/mdn/yari/actions/runs/5429890019 completed and the revert has now been deployed to production. Make sure to force-reload or otherwise clear your browser cache, to make your browser to re-fetch and get a fresh response.

[sideshowbarker]

For what it’s worth, please at least accept my personal apology as someone who, if I’d been paying more attention over the last few weeks, might’ve been able to have helped wave them off from doing this.

Apparently, there was a blog entry posted at https://developer.mozilla.org/en-US/blog/introducing-ai-help/ on June 27th, and there were some communications as early as a month ago that they had something in the works. But whatever anybody else might have heard about this ahead of time, I don’t think anybody outside Mozilla knew or anticipated quite what form this was going to take.

I do plan on doing everything I can to help make sure both that this thing never makes its way back into MDN in production, and also to help make sure that no other thing like this can ever happen again.

[placoderm]

Apparently, there was a blog entry posted

Here is the survey link from that article: https://survey.alchemer.com/s3/7405739/MDN-AI-Help

[dracos]

Thanks for this, @sideshowbarker - following on from https://github.com/mdn/yari/issues/9208#issuecomment-1615667888, the general "AI help" includes the same "errors" (I quote because error can in some way imply it knows what it's doing), e.g. this question's answer has mangled together Array.prototype.with() with the deprecation from the with page, both giving an example and then saying don't use it:

[robinwhittleton]

I appreciate that the remaining tidy-up is less urgent, but there’s still a banner and menu item proudly advertising “AI Help” that now do nothing. Can these be “paused” too?

[yretenai]

I don't understand why this exists at all in this current implementation. Looking at the AI related code, it seems to generate embeddings (which should prevent this) but doesn't actually use it? Supabase (the firebase alternative that's responsible all this AI nonsense) also does not seem to use their own embeddings?

Very strange and I'm very disappointed. All of these problem show that it was just slapped together in a day with little to no real testing or vetting, which even considering that AI is wholly counter-intuitive for MDN makes me concerned for the MDN as a whole.

[Leo40Git]

@robinwhittleton AI Help is still functional. IMO it should be paused too, but this issue is specifically about AI Explain.

[gbj]

As the author and maintainer of a web framework with a significant userbase, I think I can speak for many open-source maintainers when I say that over the last 6 months or so, we've seen a huge influx of user questions, issue reports, etc. that are basically along the lines of "Why doesn't this code work?" when "this code" is an API that simply does not exist in our library and has been hallucinated by ChatGPT without the user posting the question bothering to check the docs.

I love and rely on MDN to the extent of directly linking to MDN pages from our docs. It is the go-to source for authoritative information about the basic behavior of the Web. I am worried that if this feature is restored, it will only amplify the burden of combating misinformation about Web APIs by making ChatGPT-style hallucinations seem even more authoritatively wrong.

MDN provides amazing information that is a huge help to Web developers. Please don't clog up my Discord notifications and GitHub issues by starting to provide misinformation that is so wrong that people tell me my work has bugs!

[resuna]

The whole point of GPT and every other AI-research spinoff oriented towards deceiving readers and testers is a result of the unfortunate trend - almost since Turing wrote his imitation game paper - of taking the Turing test as a primary goal to be solved. If you spend decades building code to fool people into accepting lies (this computer program is a person, for example), you get very good code for fooling people into accepting lies. If you then have it generate "real seeming" documentation with no mechanism even conceived of for ensuring the truth of the documentation, what do you think you'll get?

[kaystrobach]

Please disable that. MDN is documentation! Documentation should be proven and bulletproof correct. See Stackoverflow guidelines which are disallowing AI content for a reason.

[mousetail]

Please disable that. MDN is documentation! Documentation should be proven and bulletproof correct. See Stackoverflow guidelines which are disallowing AI content for a reason.

Stack overflow is changing it's rules too: https://meta.stackexchange.com/q/389582/371052 It will likely become impossible for moderators to remove AI content there too

[kaystrobach]

Anyway there is a difference between documentation, and a collection of questions which get up or downvote answers. And yes AI is also a threat to stackoverflow.

[IBBoard]

Just read the latest edition of Mozilla Hacks and it looks like it's something that they were trying to push as a big positive:

[atomiks]

Maybe in 2 to 5 years this will change, but to me the tech isn't ready yet for integrations like this imo. The current trend of racing to shoehorn LLMs into every product seems a bit misguided to me, and the results show they're still too experimental for production use.

In terms of feedback for integrating this feature in the future: the current set of LLMs seem best when used as reasoning engines for analyzing input text to avoid confabulation; if this were to be integrated well, it should be fed the entire human-written MDN doc itself (rather than what seems to only be the code snippet?) with a better system prompt that tells GPT it's a factual document, and answer questions with that prerequisite. It seems it was integrated poorly in this sense because it's arguing against the facts based on the screenshots provided in the thread? Or maybe that's just a hard limitation — I'm not familiar enough with the tech.

Either way, GPT-3.5 doesn't appear smart enough to be relied upon in the first place in comparison to 4, so rushing to integrate this seems premature.

[WebReflection]

MDN now also promote stuff on top ... I mean, you had free contributors with verified content to date (I've recently contributed too, but I've contributed for years) and you spend money for an AI service that misleads users and ruins trust and make contributors also less keen to be contradicted by such AI? This is a slippery slope to me and I think, beside some manager happy to show how "modern and cool" MDN is these days, you can save that money (if spent) and erase that "feature" from likely the most important Web reference site the Internet has these days.

Anyone can ask misleading info to ChatGPT these days, let's please keep serious content serious, thanks for listening.

P.S. if W3School would do the same, what are reference sites even competing on these days? the visual layout and "who cares about the content?" :disappointed:

P.S.2 it should be the other way around ... AI should give you money if the provided content comes out of the box from your verified source of information ... maybe flip the table and move happily ever after forward with that.

[richardkazuomiller]

As @IBBoard said, it's important to understand that LLMs do not know what they don't know, which makes them inherently untrustworthy. That doesn't necessarily mean that they're useless for teaching people things. I think a valid way to use an LLM in the process of writing documentation would be to ask it to explain the code, have a human who knows what the code does review it, and then decide to add that explanation to the documentation if it is correct. If LLMs are to be used for documentation, they should be used as tools for documentation authors, not replacements.

[VulpineAmethyst]

The only niche an LLM can fill is that of writing boilerplate. It cannot explain anything. It cannot illuminate anything. It cannot synthesize. All of these tasks require knowledge. LLMs cannot know anything. They can only predict the next word to fit a prompt.

This issue should be closed to non-contributors, IMO.

[Be-ing]

Pausing due to "controversy" is hardly enough, and frankly belittles the issue and the community. This should be removed due to being useless and potentially harmful. I cannot believe this feature was even greenlit and shipped. I fear this is just another step in Mozilla's death spiral.

Indeed, @fiji-flo's commit message is not helping:

Due to a controversial disscussion of this feature we're pausing it not. We'll look into it after the weekend.

There were two pull requests (#9209, #9210) opened to revert adding this nonsense. Instead @fiji-flo chose to merely comment out two lines adding this antifeature to the UI and adding a comment hinting that it might come back. That is not being accountable; it's caving due to immediate pressure and obviously going to keep the controversy going.

[pygy]

@sideshowbarker I wish MDN became the Apache Developer Network at this point (or became independent as its own non-profit).

Mozilla has lost its way.

[Qix-]

it's caving due to immediate pressure and obviously going to keep the controversy going.

Let's be fair here. It's the weekend. This is not so pressing that it needs to have all-hands-on-deck right this very minute. Let's calm down and be fair to the people involved, shall we?

The commit mentions taking a look after the weekend. Let's let them have their investigations and discussions. Nothing further said in this thread against the AI explainer is going to add to the conversation - I think they get the picture this is widely disliked, especially given the 1k downvotes on the OP.

Simmer down a bit. It's worth assuming that nobody at play here is malicious in nature and there is a perfectly human and understandable reason for the chain of events happening here. No need to react so vitriolically with some of the comments here calling for the person to be fired, etc.

[airbr]

I just had to post that I paid for MDN Web Docs plus as a subscriber for several months while I could justify affording it, and I liked many features of it, and I had to cut back but I don't want to go back to subscribing if this AI feature is the direction or strategy of this website in general.

People are reacting strongly because they actually really use and value this website and are concerned by trends like this.

As someone who has read the website a lot I feel like, I can definitely say, this feature feels like spam.

Edit: To further explain what I mean, its like Im reading a website around this ai content that has context and authorship and a date and time and its something I can readily work to fix. This AI stuff is written at some kind of lower grade level that doesnt take into account context. It feels like someone basically, talking real slow, when Ive picked up enough from the rest of the website to know what I should? I dont know. Its a lot of words and low quality on the page.

[nyeogmi]

Why is the AI Help button still here? This is gross.

[IBBoard]

Why is the AI Help button still here? This is gross.

@nyeogmi - See https://github.com/mdn/yari/issues/9208#issuecomment-1615828812

AI Help is still functional. IMO it should be paused too, but this issue is specifically about AI Explain.

(Emphasis mine)

[nyeogmi]

Why is the AI Help button still here? This is gross.

@nyeogmi - See #9208 (comment)

AI Help is still functional. IMO it should be paused too, but this issue is specifically about AI Explain.

(Emphasis mine)

I agree with your literal explanation. Let me restate my comment: what good reason is there for the AI Help button to be here? It should be removed.

[babolivier]

@nyeogmi - The point @IBBoard is making is that this is not the place to discuss AI Explain. This issue already has close to 100 comments, and making the discussion diverge by starting to discuss points that aren't strictly in the scope of this issue is not helping and will likely just result in a blob of noise that will not bring any value to anyone (which imo this thread is already getting close to on its own). I'm not saying discussing AI Explain is a bad idea, but that this discussion should probably happen in its own issue instead of hijacking this one.

[nyeogmi]

@nyeogmi - The point @IBBoard is making is that this is not the place to discuss AI Explain. This issue already has close to 100 comments, and making the discussion diverge by starting to discuss points that aren't strictly in the scope of this issue is not helping and will likely just result in a blob of noise that will not bring any value to anyone (which imo this thread is already getting close to on its own). I'm not saying discussing AI Explain is a bad idea, but that this discussion should probably happen in its own issue instead of hijacking this one.

You are right. Here is a new issue.

[Akselmo]

Nothing like a technical information site lying to me when I'm looking for something. Well done, I guess.

[Adamkaram]

i'm wondering if it's that bad how this AI would deal with Webrtc API information

[Be-ing]

So the "solution" is adding a disclaimer and a survey instead of removing the false information? :upside_down_face: :upside_down_face: :upside_down_face: Every change to the MDN website so far has been doubling down while pretending to listen. It's starting to look like a fork may be warranted...

[MrLightningBolt]

So either someone is paying Mozilla to do this, or it's being recklessly championed by someone who cares more about getting on the AI hype train than on MDN being a reliable source of information, and in either case it means MDN is effectively compromised in such a way it can no longer be considered trustworthy.

[Zarthus]

I would have liked to see more interaction from the authors (@LeoMcA, @fiji-flo) in this thread rather than just a merge request that was approved & merged within 12 minutes of inception.

It feels to me like @sideshowbarker actually understands and is just picking up/doing damage-control while other maintainers (still) are just doing their own thing without showing the accountability or fully understanding the problem.

It's totally okay to let a merge request linger for longer than 15 minutes and garner some feedback, especially if you have recently made a mistake and seek to remedy that when the firefighting is already done. Getting it out the door ASAP kind of just comes across untrustworthy or rushed to me.

To reiterate what was said 2 days ago by Sorixelle:

https://github.com/mdn/yari/issues/9208#issuecomment-1615630693

Glad it's partially reverted for now, but... commits directly to main, without so much as a PR, or even a message in this issue we have for tracking this? Sounds like Mozilla has some serious accountability concerns to address.

I hope MDN writes a public post-mortem/blog once this is all more thoroughly discussed in depth by the internal team and have had time to retrospect.