Closed cayb0rg closed 3 days ago
This is amazing, as discussed:
null
instead of randomly generated, to prevent collisions.gitignore
Some additional thoughts:
install.yaml
, would it also be possible to add custom prompts, to better address variations in individual widget engines? Would this improve some of the compatibility issues with certain widgets?Thanks for the feedback! Included some fixes for the things we discussed.
We might want to consider a variant of the Save History Keep/Cancel action bar overlay for this feature, to address that this is the generated qset and to highlight that if the content didn't load correctly (because the creator encountered an error with the generated qset, etc) the user can cancel and go back
Didn't even notice this, haha. We're definitely not "Previewing Prior Save". Apart from changing the title, are you suggesting changing "Select Cancel to go back to the version you were working on. Select Keep to commit to using this version." and/or altering the behavior of the "Cancel" button?
Assuming we add a flag in the install.yaml, would it also be possible to add custom prompts, to better address variations in individual widget engines? Would this improve some of the compatibility issues with certain widgets?
This could work for sure!
Didn't even notice this, haha. We're definitely not "Previewing Prior Save". Apart from changing the title, are you suggesting changing "Select Cancel to go back to the version you were working on. Select Keep to commit to using this version." and/or altering the behavior of the "Cancel" button?
Behavior of keep and cancel should probably be the same, I'm just thinking of adding some additional language to clarify that the creator loaded the questions generated by AI, and that if the creator did not load properly you can hit cancel to return to where you were. Something to that effect, since the likelihood of a creator-breaking qset is higher.
Added custom prompts to the install.yaml
for widgets, which fixed the problem This or That was having! I also added a custom confirmation dialog and a bit of additional language to the question generator dialog.
Widget_Question_Generator
class.provider
generation configuration is set to openai
or azure_openai
, in order to support both options.is_generable
API endpoint and bundled the is_generable
flag as a widget class property. The property is a composite of the widget's is_generable
flag and the system's global flag for question generation.is_generable
flag returns true. Widget engines must opt in for question generation via the is_generable
property in their install.yaml
.is_generable
flag is now globally disabled when a user is not authenticated or doesn't have the basic_author
role.env
values from .env
and .env.local
respectively instead of explicitly listing them. Allows developers to put local secrets in .env.local
without worrying they will be committed and added to the git history.is_generable: Yes
property to a widget's install.yaml
onSaveClicked(mode)
method: if mode
is 'history'
, validation should be bypassed.generation_prompt
field to the meta_data
section of a widget's install.yaml
. This prompt should provide clarifying instructions to facilitate the proper and consistent formatting of a qset.
Up-to-date details of this PR available here:
https://github.com/ucfopen/Materia/pull/1582#issuecomment-2302648824
Qset Generator
Uses OpenAI's gpt-3.5-turbo model to generate a qset based off the widget's
demo.json
structure and the dall-e-2 model for generating images.Setup
Generate an OpenAI key and add it to your
.env
file like:Run Materia as usual. Note: this PR adds the openai-php/client package with composer which may require a Materia reinstall for those who have it installed already.
Create a widget. The instance has to have an existing ID to work, so new instances must be saved first before using the generator.
Click on "Generate Questions" in action toolbar. Enter a descriptive topic (3+ words at least). You may also choose to extend the current qset so that the current questions are kept and include images. Including images will take much longer. See widget compatibility for what is supported.
Appending to the current qset doesn't get the exact number right all the time. It will also take longer than generating a new qset with the same number of questions.
A log file called
openai_usage.txt
will be generated and stored in the/public
folder with the generation times, tokens, costs, and more. I've attached an exampleopenai_usage.txt
from testing. Prompt tokens are for the input prompt, which includes the demo qset, your topic, and additional instructions. Completion tokens are for the output, or what the model used. openai_usage.txtgpt-3.5-turbo costs:
Widget Compatibility
This list will be updated as individual widgets are tested.
install.yaml
): 'Each answer is stored in the answer's options > asset. Inside asset, set the 'materiaType' attribute to 'text', the 'type' attribute to 'text', and the 'value' attribute to the actual answer.'More on images...
Generating images is pricey, at $0.016 per 256x256 image. The time to generate these is even more exorbitant; however, you can still test it if you'd like on This or That. Also, OpenAI will yell at us if it tries generating images with real people in it, which may happen by accident if the questions it itself generated had real people's names.
Furthermore, image generation is not compatible for most widgets, mainly because widgets store their assets in weird places which make them hard to track and manipulate. Therefore, images will most likely never make it to production, but feel free to try it out!
To-Do
install.yaml
on generation compatibility. Disable in creator if not compatible.install.yaml