search

TogetherCrew / airflow-dags

1 stars 1 forks source link

Add Forum retriever background job processing #28

Closed amindadgar closed 10 months ago

amindadgar commented 10 months ago

We need to add the forum retriever to the background job processing as we need to answer a huge number of user requests. We could either go with redis or Celery but additional research must be done in case of background job monitoring.

amindadgar commented 10 months ago

This issue will be resolved using the hivemind-bot repo using the Celery app as it supports concurrent job processing. https://github.com/TogetherCrew/hivemind-bot

Another issue is already defined for this in hivemind-bot repo https://github.com/TogetherCrew/hivemind-bot/issues/2