BharatSahAIyak / autotune

A comprehensive toolkit for seamless data generation and fine-tuning of NLP models, all conveniently packed into a single block.
MIT License
8 stars 5 forks source link

Migrate Autotune to Django #31

Closed KDwevedi closed 4 months ago

KDwevedi commented 4 months ago

Concurrent Calls to Open AI not working.

@sooraj1002 Tried the following approaches:

  1. Just using django with asyncio. This made the request synchronous as django by itself is synchronous.
    • No Support for background tasks in django
  2. the original approach by radhay, but that was just adding doing sequential processing for each batch
  3. tried using threadpoolexecutor, but for that, we would have to define the hardware we have.
  4. celery + asyncio doesnt have support, otherwise it wouldve been the best option as asyncio works on a single thread.

Sooraj talked to @ChakshuGautam, have to implement a synchronous call w single celery worker.

KDwevedi commented 4 months ago
KDwevedi commented 4 months ago

bg task: https://django-background-tasks.readthedocs.io/en/latest/ async calls: https://docs.djangoproject.com/en/5.0/topics/async/

KDwevedi commented 4 months ago