AI bugs - Githubissues

kof commented 1 year ago

Fix before launch

[x] speech always adds a space before prompt, needs a trim
[x] after there was some error - we need to cancel everything AI bar is doing and return it to initial state
[x] disable deleting body with "delete instance" prompt
[x] double-sends AI requests
[x] the AI commands bar should not be visible when Preview Mode is enabled
[x] when no instance selected we should either show this in AI bar or automatically assume body
[x] show user friendly errors in the toast, e.g. right now FUNCTION_INVOCATION_TIMEOUT shows "Unknown error" to the user
[x] edit-styles operation - explore adding instructions about using square brackets notation when fulfilling custom style editing requests that are not possible with supported Tailwind classes

Fix later

One thing worth keeping in mind is that at this stage it is only worth fixing core bugs, not trying to get better results or fix one offs (eg. the AI doesn't make a the background black with this prompt).

If you start to fix one offs you will likely break other things. Substantial improvements will have to come by overall improvements as described in "Future features" and "Core Architecture Improvements" here https://github.com/webstudio-is/webstudio/issues/2376

taylornowotny commented 1 year ago

I can write human error messages for the toasts if you list the possible errors for me.

kof commented 1 year ago

@istarkov @gsppe @TrySound Let's aggregate a full list of error codes we need to render human readable messages for, edit this list and give it an explanation in your words

FUNCTION_INVOCATION_TIMEOUT - API took too long and we decided to stop the request. I think we probably need to tell user to try again? @gsppe wdyt we should tell user in such cases?
ai.invalidApiKey (when no pexels key is set)

gsppe commented 1 year ago

I increased the running limit to 3 minutes, not sure if you tried before or after that. Maybe we can tell that the request took too long?

itsNintu commented 1 year ago

I'm running into the "something went wrong" error when my prompt starts to get into detail.

For example:

"Create a hero section" - This works fine.
" Create a hero section with two columns. On the left column, we will have a heading, a description, and a call to action button. And on the right column, we will have an image." - This fails.

When it fails I get the following console error:

Failed to load resource: the server responded with a status of 504 ()
builder.$projectId-OSZEMVGX.js:49 Error: Something went wrong
    at vN (builder.$projectId-OSZEMVGX.js:48:15805)
    at async D (builder.$projectId-OSZEMVGX.js:49:1750)
D @ builder.$projectId-OSZEMVGX.js:49

Tested on : https://webstudio-builder-git-pexels-webstudio-is.vercel.app/builder/c29a09ba-e17d-4af8-bc6a-a5a800c63c29?pageId=QVfs9pPAOPhsYBbD3vsUd

gsppe commented 1 year ago

@itsNintu this might be due to the fact that you have selected the entire page and therefore we are sending a context that is too big and exceeds the OpenAI token limits per message. Can you try to select a smaller element or create an empty Box, select that and try your prompt again?

itsNintu commented 1 year ago

@gsppe thanks!

I gave it a quick try by adding an empty box, selecting it and then asking AI but unfortunately I run into the same error.

gsppe commented 1 year ago

@itsNintu ok thanks, will give it a try

itsNintu commented 1 year ago

When AI uses grid, could we add, row and column values to the style source ui? The rest of the options are already there, adding these two would make it so users can also customize grids generated by ai.

kof commented 1 year ago

I didn't know we even have this grid support in that form at all!

That's def. doable before we get a proper UI for grid cc @taylornowotny

itsNintu commented 1 year ago

Yeah also pleasant surprise to me too this morning.

Honestly, if we added the above and gap options I think it covers 90% of grid usage already, I would love this and get it early over waiting for a proper grid UI.

kof commented 1 year ago

[ ] it tried to put instances into an Image

gsppe commented 1 year ago

it tried to put instances into an Image

@kof this should probably be a generic fix for Webstudio, not exclusively related to this feature. Bogdan might have ides of how to go about this

TrySound commented 1 year ago

findClosestDroppableTarget is generic fix but kinda doesn't fix into ai logic. You can try to reuse it's internal findClosestDroppableComponentIndex

itsNintu commented 1 year ago

With the latest prod deployment, I am getting 2x what I'm asking for. Demo: https://www.berrycast.com/conversations/968b782e-03da-59b8-b00a-e48155424cc5

Prompt: full-width hero image showcasing the cozy and inviting atmosphere of the cafe with overlaying text in the center. The headline should be compelling and welcoming, followed by a subheading that briefly describes what makes The God Shot special.

itsNintu commented 1 year ago

Getting a new error with more complex prompts:

Console: Unchecked runtime.lastError: The message port closed before a response was received.

Network:

itsNintu commented 1 year ago

In trying to get it to generate a background image it fails but does not tell me why. Nothing seems to happen.

This is on prod

Demo: https://www.berrycast.com/conversations/b716ed9d-411c-535d-b20f-aefa0e347c7d

gsppe commented 1 year ago

Background images (css background images) are not supported. We could add a feature detection for this kind of task and say that it is not supported, but then I wonder how many others one-off cases we should support and how adding those to the feature detection endpoint might mess up feature detection/interfere with other more generic features

itsNintu commented 1 year ago

Thanks @gsppe.

Here's another one, AI seems to always target the box that is contained when asking to change the background color (even though I have selected the correct div)

https://www.berrycast.com/conversations/65445c85-b0d6-5577-9f2e-b4c3e52e88b2

gsppe commented 1 year ago

@itsNintu try "make the outer container black"

itsNintu commented 1 year ago

Prompt: Create a hamburger menu using radix sheet component

Returns: "Error Could not get meta for component "@webstudio-is/sdk-components-react-radix:Sheet""

Console: Error: Could not get meta for component "@webstudio-is/sdk-components-react-radix:Sheet" at h2 (builder.$projectId-FKMCE2TI.js:33:13953) at b2 (builder.$projectId-FKMCE2TI.js:33:17674) at du (/build/_shared/chunk-UICIJ4KI.js:6:19466) at Ba (/build/_shared/chunk-UICIJ4KI.js:8:43659) at Ua (/build/_shared/chunk-UICIJ4KI.js:8:39459) at Vf (/build/_shared/chunk-UICIJ4KI.js:8:39390) at qr (/build/_shared/chunk-UICIJ4KI.js:8:39250) at Qo (/build/_shared/chunk-UICIJ4KI.js:8:36419) at pn (/build/_shared/chunk-UICIJ4KI.js:6:3250) at /build/_shared/chunk-UICIJ4KI.js:8:34024 c @ /build/_shared/chunk-EHX7Y6UX.js:1 Error: Could not get meta for component "@webstudio-is/sdk-components-react-radix:Sheet" at h2 (builder.$projectId-FKMCE2TI.js:33:13953) at b2 (builder.$projectId-FKMCE2TI.js:33:17674) at du (/build/_shared/chunk-UICIJ4KI.js:6:19466) at Ba (/build/_shared/chunk-UICIJ4KI.js:8:43659) at Ua (/build/_shared/chunk-UICIJ4KI.js:8:39459) at Vf (/build/_shared/chunk-UICIJ4KI.js:8:39390) at qr (/build/_shared/chunk-UICIJ4KI.js:8:39250) at Ii (/build/_shared/chunk-UICIJ4KI.js:8:35673) at Qo (/build/_shared/chunk-UICIJ4KI.js:8:36474) at pn (/build/_shared/chunk-UICIJ4KI.js:6:3250) c @ /build/_shared/chunk-EHX7Y6UX.js:1 Error: Could not get meta for component "@webstudio-is/sdk-components-react-radix:Sheet" at h2 (builder.$projectId-FKMCE2TI.js:33:13953) at b2 (builder.$projectId-FKMCE2TI.js:33:17674) at du (/build/_shared/chunk-UICIJ4KI.js:6:19466) at Ba (/build/_shared/chunk-UICIJ4KI.js:8:43659) at Ua (/build/_shared/chunk-UICIJ4KI.js:8:39459) at Vf (/build/_shared/chunk-UICIJ4KI.js:8:39390) at qr (/build/_shared/chunk-UICIJ4KI.js:8:39250) at Ii (/build/_shared/chunk-UICIJ4KI.js:8:35673) at Qo (/build/_shared/chunk-UICIJ4KI.js:8:36474) at pn (/build/_shared/chunk-UICIJ4KI.js:6:3250) xi @ /build/_shared/chunk-UICIJ4KI.js:8 React Router caught the following error during render Error: Could not get meta for component "@webstudio-is/sdk-components-react-radix:Sheet" at h2 (builder.$projectId-FKMCE2TI.js:33:13953) at b2 (builder.$projectId-FKMCE2TI.js:33:17674) at du (/build/_shared/chunk-UICIJ4KI.js:6:19466) at Ba (/build/_shared/chunk-UICIJ4KI.js:8:43659) at Ua (/build/_shared/chunk-UICIJ4KI.js:8:39459) at Vf (/build/_shared/chunk-UICIJ4KI.js:8:39390) at qr (/build/_shared/chunk-UICIJ4KI.js:8:39250) at Ii (/build/_shared/chunk-UICIJ4KI.js:8:35673) at Qo (/build/_shared/chunk-UICIJ4KI.js:8:36474) at pn (/build/_shared/chunk-UICIJ4KI.js:6:3250) Object

kof commented 1 year ago

@itsNintu reproduced it too @gsppe any chance you can fix it?

itsNintu commented 1 year ago

Also getting an error when trying to use the radix navigation menu:

"Error NavigationMenuContent must be used within NavigationMenuItem

Prompt: Create a navigation bar with two columns, the left column has a link and an image inside of it, the right column has a radix navigation menu and CTA link beside it.

kof commented 1 year ago

@itsNintu I reproduced it too @gsppe @TrySound any ideas? The problem is that some components have requirements to be used as children of some other components, how can we enforce this?

itsNintu commented 1 year ago

I think AI broke the publishing function of my site.

When trying to publish I get "Publish Failed"

When visiting the site it says "Worker not found"

Site url: https://waitlist-222222.wstd.io/