search

langgenius / dify

Dify is an open-source LLM app development platform. Dify's intuitive interface combines AI workflow, RAG pipeline, agent capabilities, model management, observability features and more, letting you quickly go from prototype to production.
https://dify.ai
Other
46.12k stars 6.5k forks source link

Is it possible to change the dify app execution engine from Celery to another workflow engine, e.g. argo-workflow ? #6935

Open Colstuwjx opened 1 month ago

Colstuwjx commented 1 month ago

Self Checks

1. Is this request related to a challenge you're experiencing? Tell me about your story.

Hi team,

Thanks for sharing this awesome project! We're evaluating if we can use Dify in our production environment, however, it seems like that we need to support some production requirements in advance:

  1. since it's production environment, it's necessary to have a way to do canary deployment and also traffic control, rate limit, I've raised an individual issue for this, see ref #6893
  2. we need to build a CICD pipeline to evaluate the GenAI app quality by running several golden tests and also manage the released versions by a more robust way, e.g. DSL + GitOps way. I think it's possible for Dify since it supports to export the DSL and also import it, which means we can integrate with a GitOps release pipeline by these two apis
  3. however, since we're using Celery as the workflow execution engine, I'm afraid the isolation between each tenant might not be good enough ? How can we avoid that one bad-design app impact the other GenAI apps running on the same one Dify Cluster ? One of the solution that could address this issue could be using a CloudNative execution engine and use K8s scheduling feature to resolve it. That's why I propose this issue.

Any ideas if the point 3 could be addressed by some ways ? Thanks!

2. Additional context or comments

I've experienced the last cloud native revolution, and I strongly believe that if one project could be pluggable and involve more community power, it could adopt more end users and gain more contributions from the community. IMO, one of the success factor of Kubernetes project is that it allows to be customized by many ways, the Resource + Reconcile pattern / CNI / CSI / SideCar design allows more community developers to join the game. Dify and the other LLMOps platform could potentially to be the Next Kubernetes in the GenAI indurstry.

3. Can you help us with this feature?

Colstuwjx commented 1 month ago

BTW, just searched the current best practices of AI apps, e.g. JupyterHub offered a way to integrate with Kubernetes: https://jupyterhub.readthedocs.io/en/stable/explanation/concepts.html#spawner

crazywoola commented 1 month ago

Hello @Colstuwjx

Thank you for your insightful suggestion and interest in the Dify project. We appreciate the enthusiasm and the potential contribution you've expressed.

Regarding your proposal to consider updating or replacing the Celery Worker dependency with another workflow engine such as argo-workflow, we have taken it into serious consideration. We understand the importance of features like canary deployment, traffic control, and rate limiting in a production environment, as well as the benefits of using a CloudNative execution engine with Kubernetes scheduling for better tenant isolation.

However, at this moment, we do not have plans to update or replace our dependency on Celery in the near future. Celery serves as a robust and widely-adopted workflow execution engine that plays a significant role in our current Dify architecture. We will continue to monitor and assess its performance to ensure it meets the needs of our users.

Nevertheless, we truly value your forward-thinking and suggestions for the future development of the project. We believe that with the growing community and contributions like yours, Dify has the potential to become a leading platform in the GenAI industry. We encourage and welcome more community members to get involved in the project, driving its progress and evolution.

Should you have any further ideas or suggestions, or if you are willing to contribute to this feature, please do not hesitate to get in touch to @takatost . We look forward to working with you to build an even stronger and more adaptable Dify platform.

Thank you once again for your contribution and support!

Colstuwjx commented 1 month ago

Hi @crazywoola , thanks for your kindly reply.

After deep diving into the sourcecode, I suddenly found the Dify workflow execution is actually triggered by a sync api, and also use an original new Thread way rather than Celery workers. The entrance http api is workflow/run and the api controller would execute by this process: AppGenerateService -> WorkflowAppGenerator (new Thread here) -> WorkflowAppRunner -> WorkflowEngineManager.

Related discussion issue: ref #4489. Thanks.

Colstuwjx commented 1 month ago

I've got an idea to resolve this issue, by taking advantage of Serverless Engine, e.g. OpenFaas. The details be like:

  1. when someone published an app on Dify platform, it could trigger a GitOps pipeline to generate the serverless function code and its corresponding workflow DSL, the published app would be regarding as an individual microservice
  2. The individual dify app would be running on OpenFaas, e.g. by this way: https://www.openfaas.com/blog/openfaas-flask/
  3. Therefore, Dify doesn't need to reinvent the wheels like API Gateway, CICD pipeline etc, just use the industry leading solutions

Any suggestions ?

lilyzhoupeijie commented 1 month ago

@Colstuwjx hi,we also want to publish dify workflow to argo ,do you have interest in working together?