dotCMS / core

Headless/Hybrid Content Management System for Enterprises
http://dotcms.com
Other
865 stars 467 forks source link

Implement AI image generation for Binary fields #27458

Closed john-thomas-dotcms closed 3 months ago

john-thomas-dotcms commented 9 months ago

The AI plugin currently supports generative AI in the block editor, including creation of AI-generated images.

We need to duplicate the image generation capabilities of the AI Image block for the standard Binary field in the content editing screen.

User Story

As a Content Architect in dotCMS, I want to generate a image with AI in the Binary Field.

image

Screenshot 2024-06-17 at 11 19 51 AM

Acceptance Criteria

github-actions[bot] commented 6 months ago

This issue is stale because it has been open 90 days with no activity. Remove stale label or comment or this will be closed in 30 days.

oidacra commented 4 months ago

Waiting for https://github.com/dotCMS/core/issues/28703 Spike

nicobytes commented 4 months ago

QA Note: trunk_7e09771

Preconfig

  1. Create a new content type with Edit Content Beta Screenshot 2024-07-03 at 3 06 26 PM
  2. In this new schema, select BinaryField and BlockEditor Screenshot 2024-07-03 at 3 07 31 PM
  3. Enable "Allow users to create a file by AI" in the binary field setting. Screenshot 2024-07-03 at 3 08 13 PM

Case 1:

The button should not be in the options if this option is off in the binary field settings.

Binary settings Binary field
Screenshot 2024-07-03 at 3 15 43 PM Screenshot 2024-07-03 at 3 16 03 PM

Case 2:

The button should be disabled if dotAIApp is not installed but the ai option is enabled in the binary field settings.

Binary settings dotAIApp Binary field
Screenshot 2024-07-03 at 3 08 13 PM Screenshot 2024-07-03 at 3 22 18 PM Screenshot 2024-07-03 at 3 12 55 PM

Case 3:

The button should be enable if dotAIApp is installed but the ai option is enabled in the binary field settings.

Binary settings dotAIApp Binary field
Screenshot 2024-07-03 at 3 08 13 PM Screenshot 2024-07-03 at 3 24 42 PM Screenshot 2024-07-03 at 3 25 11 PM

Case 4: Generate image from AI

  1. Click on "Generate with dotAI".
  2. The dialog opens
  3. The dialog should have only one option "Create with Custom Prompt" because the option "Create based on existing content" should not be present for binary fields.
  4. Choose an orientation
  5. Write your prompt for creating an img, e.g: "A cat eating a burger".
  6. Click the Generate button
  7. Click the Insert button
  8. Save the new content
  9. Reload the page
  10. The image should remain because it has already been saved.

https://github.com/dotCMS/core/assets/7611944/ac600b54-dcd0-456a-8dc7-2bbbb012227f

Case 4: Interoperability with Block Editor

The idea is to use Generate AI images from different fields without problems.

  1. Generate image with AI from Binary Field
  2. Generate image with AI from Block Editor Field

https://github.com/dotCMS/core/assets/7611944/43922743-4750-4ffa-982d-93195eb839c8

Case 5: Binary field should still work in old editor and new editor

Here is a video showing the binary field working in the old editor and the new editor

https://github.com/dotCMS/core/assets/7611944/287ca76e-5b6d-4e6c-a052-1563d56dc0be

hmoreras commented 4 months ago

IQA scenarios tested:

Based on Nico's comment:

Case 1, 2 and 3 in the old and new editor: Pass IQA. Cases 4 and 5: pass IQA.

Couple of details notice:

  1. Links and buttons in the binary field should be the same as the others fields .

    image
  2. There is a small jump when you open the ai image dialog from the binary field, do not happen from the block editor.

https://github.com/dotCMS/core/assets/31667212/4d27361e-4ed3-4011-a49f-6b55be97e1bc

nicobytes commented 4 months ago

Fix the problem with the colors and other bugs, but in this case: There is a small jump when you open the ai image dialog from the binary field, does not happen from the block editor. I need to solve this issue in another PR because I need more research about this behavior.

hmoreras commented 3 months ago

QA note: feedback 2 in my last comment will be worked here: https://github.com/dotCMS/core/issues/29173

josemejias11 commented 3 months ago

Approved: Tested on trunk_430f4c7, Docker, macOS 14.5, FF v126.0.1