search

Neural-Dragon-AI / Cynde

A Framework For Intelligence Farming
Apache License 2.0
12 stars 0 forks source link

Move oai embeddings to work with asynclient and json cache and autobatching #1

Open furlat opened 4 months ago

furlat commented 4 months ago

Embedding with open ai is currently very naive we need to add

  1. Embeddings are currently called on the main python trhead with a for-loop it should be moved to work in the same way as column generation
  2. OpenAi client allows "batched" calls to the api client hence we should think about this in the balancing
furlat commented 4 months ago

Almost solved in https://github.com/Neural-Dragon-AI/Cynde/pull/11 only missing integration in embedd_column

furlat commented 4 months ago

We need to produce one example where we attempt to embed at rate limit for a 1 minute task.