cloudflare / cloudflare-docs

Cloudflare’s documentation
https://developers.cloudflare.com
Creative Commons Attribution 4.0 International
2.69k stars 2.75k forks source link

Update or retire Cursor AI assistant #14773

Open Cherry opened 1 month ago

Cherry commented 1 month ago

Existing documentation URL(s)

https://developers.cloudflare.com/workers/ai/

What changes are you suggesting?

The information provided by Cursor is often outdated or incorrect, and needs to be updated, or simply retired.

Some examples:

Is Workers AI stable? Workers AI is currently in Open Beta and is not recommended for production data and traffic. Limits and access are subject to change. SOURCE: workers-ai/

Workers AI was announced as GA back in Developer Week.


When should I use Workers Sites vs Cloudflare Pages? For hosting full-stack applications, it is recommended to use Cloudflare Pages instead of Workers Sites. Workers Sites is suitable for deploying static applications directly to Workers, especially those built with static site generators like Hugo. These sources might provide additional context: https://developers.cloudflare.com/workers/configuration/sites/

Pages is the recommended solution for static site generators too. This is incorrect.


What's better? Module worker format, or service worker format? The decision between the Module worker format and the Service worker format depends on your specific use case and requirements. The Module worker format allows you to export a collection of files and modules, providing more flexibility in structuring your code. It also supports ES Modules syntax, which can be beneficial for organizing and managing complex projects. On the other hand, the Service worker format requires a single file to be uploaded and relies on global variables for bindings. It may be simpler for smaller projects or those that do not require the modularity provided by ES Modules. Ultimately, the choice between the two formats should be based on the complexity of your project, the need for modularity, and your familiarity with ES Modules. These sources might provide additional context: https://developers.cloudflare.com/workers/wrangler/migration/v1-to-v2/wrangler-legacy/configuration

This fails to include any information about new products that don't work without Module Workers, or that they're faster, and more secure.


What's the available pricing models for Workers? Workers are available under two Usage Models: Bundled and Unbound. Usage Models are settings on your Workers that specify how you are billed for usage, as well as the upper limits. These sources might provide additional context: https://developers.cloudflare.com/workers/platform/pricing

This is inaccurate, with Standard being the only current pricing model available for Workers.


Is D1 ready for production traffic? D1 is currently in public beta, and while the D1 team expects breaking changes and issues to be minimal, they may still occur. The D1 team generally does not recommend running large production workloads on beta products. These sources might provide additional context: https://developers.cloudflare.com/d1/

These are just a few examples where the responses are out of date, inaccurate, unhelpful, and in some cases could be potentially harmful if a prospective customer used the data. It does come with a disclaimer about accuracy, but I think some effort needs to be made to keep this up to date and not (seemingly) abandoned. Or if that's unlikely, it should probably just be retired and folks can rely on the more useful Algolia search.

Cherry commented 1 month ago

The lack of any persistent context means you can get results like this too, where it completely changes what is actually being talked about:

kodster28 commented 1 month ago

cc: @mhart for visibility

mhart commented 1 month ago

Hey @Cherry ! I appreciate the notes about certain answers being out of date, will look at improvements here.

But ultimately, let's not bury the lede:

It does come with a disclaimer about accuracy

Let's not underplay this fact – the disclaimer is quite large and still 100% relevant. Unless we move this out of an experimental phase, this will still be the most important thing to keep in mind.

Cherry commented 1 month ago

Thanks @mhart, I appreciate the followup. It's good to hear that some things will be updated so answers aren't out of date.

Let's not underplay this fact – the disclaimer is quite large and still 100% relevant. Unless we move this out of an experimental phase, this will still be the most important thing to keep in mind.

While I understand this, I think you overestimate how useful this disclaimer is, or how many people read it. More often than not, the feedback we see in Discord when someone uses this is something like "but the AI said X", and we have to tell them it's inaccurate and point them at the right documentation. A disclaimer and being experimental imo isn't an excuse to dismiss accuracy issues like this.

This is all especially true in my second reply to this issue about AI. I asked Cursor a question about AI, got an answer, then asked a followup question, expecting context to be retained. But I then got an answer completely unrelated to AI, about router models supported by magic network monitoring. Users familiar with other bots like ChatGPT or Gemini are very very used to this kind of contextual conversation where the answers are wayyy better, so are again likely to just ignore / dismiss any disclaimers. Is this really a good first impression for folks interacting with the docs, or seeing Cloudflare's AI offerings for the first time?

We've seen in the news recently too where a company had to honour what an AI chatbot said, despite it being incorrect, and despite it linking to the source documentation that did include the full policy. To play devil's advocate here for a second, if Cursor tells me that my Workers bill is going to just be $0.50 per additional million requests with no mention of other charges like CPU time with Standard for example, will Cloudflare honour that?

mhart commented 1 month ago

I've updated the index, the out-of-date answers should be a little better now. Thanks for the nudge!

Workers Sites is a tricky one, just because we do still document it. Eg https://developers.cloudflare.com/workers/configuration/sites/ does say:

Workers Sites enables developers to deploy static applications directly to Workers. It can be used for deploying applications built with static site generators like Hugo and Gatsby, or front-end frameworks like Vue and React.

So the bot still references that in its answer.

Cherry commented 1 month ago

Thanks @mhart!

I think it would be great if the index updating could be automated in a github action workflow (or something) every week or so, to ensure the data never gets quite as out of date again. What do you think?

And I think adding persistent context in conversations would really help make Cursor's responses much more helpful for most users, as showcased in my questions around AI to it in the previous reply: https://github.com/cloudflare/cloudflare-docs/issues/14773#issuecomment-2134144353