OpenAdaptAI / OpenAdapt

Open Source Generative Process Automation (i.e. Generative RPA). AI-First Process Automation with Large ([Language (LLMs) / Action (LAMs) / Multimodal (LMMs)] / Visual Language (VLMs)) Models
https://www.OpenAdapt.AI
MIT License
1k stars 138 forks source link

Extract Process API from Process Graph #586

Open abrichr opened 8 months ago

abrichr commented 8 months ago

Feature request

We wish to implement a Process API by prompting an LLM with the Process Graph and/or Screenshots/Actions, e.g. with Chain-of-Code

image image

Motivation

LLMs are great with APIs.

Related

https://github.com/OpenAdaptAI/OpenAdapt/issues/545

abrichr commented 5 months ago

https://chain-of-code.github.io/

https://colab.research.google.com/github/google-research/google-research/blob/master/code_as_policies/coc_demo.ipynb