"Bizarro" Devin Brainstorming Features
Core behaviors
- auto write code into
sketch.js
(incrementally)
- look at errors and fix bugs
- narrate its process of writing the code
- preview pane to show p5.js canvas (w/ auto-refresh)
Interaction Features
- Use whisper to listen to what I am saying
- Narrate the process, banter with me using TTS model?
ML Model
- Fine-tune a model?
- RAG with nature of code book or other CT challenge examples?
- multi-modal so it can look at the canvas
Models/frameworks under consideration
My preference is for on device and fine-tuned model.
- Gemma
- transformers.js models
- llama (via ollama)
- localAI models