okfn / opendataeditor

The Open Data Editor (ODE) is a no-code application to explore, validate and publish data in a simple way. Forever free and open source project powered by the Frictionless Framework.
http://opendataeditor.okfn.org
MIT License
183 stars 22 forks source link

Implement AI component #635

Open romicolman opened 1 week ago

romicolman commented 1 week ago

Problem description

A couple of weeks ago, we temporarily hid the AI dialog in the ODE because our team decided to first reflect on how we would make the AI integration. The process involved several team discussions guided by Madelon Hulsebos (AI consultant) to get feedback, insights and recommendations. In this process, we tried to address these key topics:

Step 1

Captura de pantalla 2024-11-06 a la(s) 2 09 27 p  m

Order: AI, Publish and Save Changes.

Button name: AI

Since the name is short, you can add the same icon we used in the past: Captura de pantalla 2024-11-06 a la(s) 3 02 23 p  m

Other specifications: because the aim of the AI integration will help users improve name and descriptions for columns, please, activate the AI button whenever a user ingest a table in csv, Excel or Google Sheets.

Step 2

After clicking on the AI button, the user adds their credentials to activate the AI feature.

Step 3

Then, the ODE displays a new dialog with this message to inform the user what will happen after validating credentials with the AI system:

If you proceed, the Open Data Editor will only share the names of the columns in your table to suggest improvements to the titles and descriptions associated with them. Do you want to proceed?

Cancel / OK

Step 4

If the user clicks on Cancel, the ODE will show the datagrid again. If the user accepts, the ODE will show the output in a pop up window:**

Note: it's up to the user to decide to edit column names and descriptions after the AI result.

madelonhulsebos commented 5 days ago

Looks good! Two comments: 1) naming the button "AI" might be too generic if there is no further description of the functionality provided (if you see "AI" in Excel, it can refer to anything). 2) if the ODE provides the column names to an LLM anyway for descriptions, it might also prompt the LLM to suggest 1) data validation rules (the checks integrated in the ODE can be provided), and 2) relevant analysis questions. The AI could then be a dropdown with "column descriptions", "data validation rules", "analysis questions". But this could be considered for another iteration, the value gain for the ODE might be significant.

romicolman commented 4 days ago

Thanks for your comments @madelonhulsebos! @roll please, discuss this today un our meeting.