Dify AWS Tool

Introduction

This repo provides some sample code to show how to integrate SageMaker Provider and some AWS Service based tools to Dify.

In addition to the reference code, you can also refer to the Dify official guide for more information.

Prerequisites

Dify Environment (It can be deployed on AWS with one click using Cloudformation Template - dify.yaml)
AWS Account and AWS Experience
Basic experience with Linux environments

Assets

[Attention]：We welcome contributions of more workflows, SageMaker models, and built-in tools. You can fork this repository and submit a merge request, please also update README.md, you need to add a new row to the corresponding table

Workflow(Demo Page)

DSL Name	Description	Link	Dependency	Owner
Term_based_translate	Translation Workflow with Term mapping Retrieval Tool	DSL	Tool(Term_mapping)	ybalbert
Code_translate	Code Transform between different Program Language	DSL		binc
Basic_RAG_Sample	simple basic rag workflow with rerank tool	DSL	Tool(Rerank)	ybalbert
Andrewyng/translation-agent	Andrew Ng's translate agent.	DSL		chuanxie
rag_based_bot_with_tts	Rag based bot which can answer with voice	DSL	Tool(TTS)	ybalbert
Marketing-copywriter	marketing copywriter	DSL		Lyson Ober
Simple_Kimi	Customized KIMI app	DSL		ybalbert
SVG_Designer	SVG Designer	DSL		李继刚

Builtin_Tools

Tool Name	Tool Type	Description	Deploy_doc	Owner
Rerank	PAAS	Text Similarity Rerank Tool	Notebook	ybalbert
TTS	PAAS	Speech synthesis Tool	Code	ybalbert
Bedrock Guardrails	SAAS	Text moderation Tool, implemented through the independent assessment API ApplyGuardrail API provided on Amazon Bedrock Guardrail.		amyli
Term_multilingual_mapping	PAAS	Word Segment/ Term mapping Retrieval Tool	Repo	ybalbert
Image Translation Tool	PAAS	Translate the text on Image	Coming	tanqy
Chinese Toxicity Detector	PAAS	A tool to detect Chinese toxicity	Comming	ychchen
Transcribe Tool	PAAS	AWS transcribe service tool (ASR)		river xie

Model_Provider

Model Name	model_type	Deploy_doc	Owner
Any Model	SageMaker\LLM	Model_hub	ybalbert
Bge-m3-rerank-v2	SageMaker\Rerank	Notebook	ybalbert
Bge-embedding-m3	SageMaker\Embedding	Notebook	ybalbert
CosyVoice	SageMaker\TTS	Code	ybalbert
SenseVoice	SageMaker\ASR	Notebook	ybalbert

[Attention]： The Dify provider of SageMaker\LLM can support most open-source models. We recommend you to reploy these models using Model_hub. It's very easy and convenient which supports model fine-tuning and deployment with no-code approach

How to Install

Below Script is only for SageMaker Model_provider and AWS Builtin Tools, you can import workflows from Web Interface. SageMaker Model_provider has already been integrated in Dify v0.6.15

Set Env Variable

dify_path=/home/ec2-user/dify #Please set the correct dify install path
tag=aws

download code

cd /home/ec2-user/
git clone https://github.com/aws-samples/dify-aws-tool/

intall code

# Part of models and tools have been integrated with dify already, no extra installation needed
mv ./dify-aws-tool/builtin_tools/aws ${dify_path}/api/core/tools/provider/builtin/
mv ./dify-aws-tool/model_provider/sagemaker ${dify_path}/api/core/model_runtime/model_providers/

build image

cd ${dify_path}/api
sudo docker build -t dify-api:${tag} .

Specify the new image for api and worker services

# Modify docker/docker-compose.yaml, Please refer the below diff
diff --git a/docker/docker-compose.yaml b/docker/docker-compose.yaml
index cffaa5a6a..38538e5ca 100644
--- a/docker/docker-compose.yaml
+++ b/docker/docker-compose.yaml
@@ -177,7 +177,7 @@ x-shared-env: &shared-api-worker-env
services:
  # API service
  api:
-    image: langgenius/dify-api:0.6.14
+    image: dify-api:aws
    restart: always
    environment:
      # Use the shared environment variables.
@@ -197,7 +197,7 @@ services:
  # worker service
  # The Celery worker for processing the queue.
  worker:
-    image: langgenius/dify-api:0.6.14
+    image: dify-api:aws
    restart: always
    environment:
      # Use the shared environment variables.

restart dify

cd ${dify_path}/docker/
sudo docker-compose down
sudo docker-compose up -d

How to deploy SageMaker Endpoint

If you want to add your LLM/Embedding/Rerank/ASR/TTS model to Dify Sagemaker Model Provider, you should deploy them by yourself in AWS/SageMaker at first.
Please refer the corresponding notebook to deploy.

Target Audience

Dify / AWS User
GenAI Developer

aws-samples / dify-aws-tool

readme