xability / maidr

Multimodal Access and Interactive Data Representation
GNU General Public License v3.0
18 stars 4 forks source link

feat: create custom GPT and use action to communicate with MAIDR API #373

Open sandeep-ps opened 5 months ago

sandeep-ps commented 5 months ago

Feature Request

Description

Create a custom GPT using GPT Builder that can understand MAIDR JSON and use Action to let it communicate with MAIDR API.

Motivation

We want GPT to understand MAIDR JSON, generate that from the SVG file, and send it to MAIDR API to generate MAIDR-enabled HTML files.

Proposed Solution

Create a custom GPT using GPT builder and give examples of MAIDR SVG and JSON so that it can learn the expected format.

Additional Context

Add any additional context or screenshots about the feature request here.

Checklist

sandeep-ps commented 5 months ago

I am working on creating a custom GPT. After providing couple of SVG examples, and the JSON format data, and other instructions related to this, it looks like the GPT is expecting text labels within the SVG element:

It appears that there are no text elements with IDs in the SVG file you provided, which is unusual for a typical bar chart SVG. Without these text elements, it's challenging to extract specific information such as the chart title, axis labels, and data values.

To proceed, I can attempt a different approach to analyze the SVG structure. This might involve looking for other elements that could potentially contain the necessary data for the MAIDR format JSON. Let's try this alternative method.

I will try to provide better instructions to the custom GPT and also try with other image file formats.

sandeep-ps commented 5 months ago

I created two Custom GPTs and shared documentation containing URLs and configuration details with the team.

sandeep-ps commented 5 months ago

Tasks after discussing with the team:

  1. Create a revised Custom GPT that uses PNG and JPEGs
  2. Find alternate means for Custom GPT to communicate with MAIDR API. This can be a base64 encoded string, but the MAIDR API expects SVG content. So, probably, some conversion needs to happen.