search

ucbepic / docetl

A system for agentic LLM-powered data processing and ETL
https://docetl.org
MIT License
1.26k stars 112 forks source link

Use OpenAI Batch Mode #20

Open shreyashankar opened 1 month ago

shreyashankar commented 1 month ago

https://platform.openai.com/docs/guides/batch

403 on Discord posted this in #feature-requests, and it's a great idea. We should change the execute method of each operator to leverage batch mode. A small challenge is that we need to make sure validations/retries still work well.

shreyashankar commented 1 month ago

I looked into this a bit more. We need to also alert the user when the pipeline has finished (since it can 1-2 days). We should finish building our UI first, then support batch mode.