Closed iplayfast closed 1 month ago
To resolve the 403 error when integrating SearxNG with Dify, follow these steps:
Check Container Status: Ensure all containers are running correctly.
docker compose ps
Inspect Logs: Check the logs of the docker-api-1
and docker-nginx-1
containers for any errors.
docker logs docker-api-1
docker logs docker-nginx-1
Resource Allocation: Ensure your Docker VM has sufficient resources allocated (minimum 2 vCPUs and 8 GB of memory).
Environment Variables: Verify that the environment variables in your .env
file are correctly configured.
Restart Services: Restart the services to apply any changes.
docker compose down
docker compose up -d
Network Configuration: Ensure that the SearxNG service is accessible from the Docker container. Replace localhost
with host.docker.internal
in the service URL.
http://host.docker.internal:8080
Nginx Configuration: A 403 error can also be caused by Nginx forwarding the service to the wrong location. First, make sure the container is running, then run the following command with root privileges:
docker ps -q | xargs -n 1 docker inspect --format '{{ .Name }}: {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}'
Find these two lines in the output:
/docker-web-1: 172.19.0.5
/docker-api-1: 172.19.0.7
Open the location where you stored the Dify source code, open dify/docker/nginx/conf.d
, replace http://api:5001
with http://172.19.0.7:5001
, and replace http://web:3000
with http://172.19.0.5:3000
. Afterward, restart the Nginx container or reload the configuration.
These steps should help resolve the 403 error when integrating SearxNG with Dify [1][2].
To continue talking to Dosu, mention @dosu.
@iplayfast Hi, this tool need to be set json format as output, do you modify the settings.yml
of searxng and restart the container? can you open this url http://127.0.0.1:8081/search?q=apple&format=json&categories=general
in the browser to see whether it is worked?
官网的文档有点问题,应该是
# remove format to deny access, use lower case.
# formats: [html, csv, json, rss]
search:
formats:
- html
- json # <-- add this line
I did set the json in the format. (before opening this issue). @dosu Following your steps this is what I get.
docker compose ps
no configuration file provided: not found
however
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e5c4b965aac3 searxng/searxng "/sbin/tini -- /usr/…" 50 minutes ago Up 50 minutes 0.0.0.0:8080->8080/tcp, :::8080->8080/tcp goofy_poincare
643a7e4133a2 nginx:latest "sh -c 'cp /docker-e…" 10 days ago Up 13 hours 0.0.0.0:443->443/tcp, :::443->443/tcp, 0.0.0.0:8088->80/tcp, :::8088->80/tcp docker-nginx-1
2f03f44c2b34 langgenius/dify-api:0.6.15 "/bin/bash /entrypoi…" 10 days ago Up 13 hours 5001/tcp docker-worker-1
eb017ab09d70 langgenius/dify-api:0.6.15 "/bin/bash /entrypoi…" 10 days ago Up 13 hours 5001/tcp docker-api-1
d2e51d56de93 postgres:15-alpine "docker-entrypoint.s…" 10 days ago Up 13 hours (healthy) 5432/tcp docker-db-1
01dd0ec3862c langgenius/dify-sandbox:0.2.1 "/main" 10 days ago Up 13 hours docker-sandbox-1
c7809d79fca6 redis:6-alpine "docker-entrypoint.s…" 10 days ago Up 13 hours (healthy) 6379/tcp docker-redis-1
f965152767be ubuntu/squid:latest "sh -c 'cp /docker-e…" 10 days ago Up 13 hours 3128/tcp docker-ssrf_proxy-1
57b017c687c1 semitechnologies/weaviate:1.19.0 "/bin/weaviate --hos…" 10 days ago Up 13 hours docker-weaviate-1
d56cc8b4cbff langgenius/dify-web:0.6.15 "/bin/sh ./entrypoin…" 10 days ago Up 13 hours 3000/tcp docker-web-1
eea4e237bc5e ghcr.io/open-webui/open-webui:main "bash start.sh" 2 weeks ago Up 13 hours (healthy) 0.0.0.0:3000->8080/tcp, :::3000->8080/tcp open-webui
so I think that's equilent.
Running migrations
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
Preparing database migration...
Database migration skipped
None of PyTorch, TensorFlow >= 2.0, or Flax have been found. Models won't be available and only tokenizers, configuration and file/data utilities can be used.
[2024-08-09 04:37:52 +0000] [1] [INFO] Starting gunicorn 22.0.0
[2024-08-09 04:37:52 +0000] [1] [INFO] Listening at: http://0.0.0.0:5001 (1)
[2024-08-09 04:37:52 +0000] [1] [INFO] Using worker: gevent
[2024-08-09 04:37:52 +0000] [96] [INFO] Booting worker with pid: 96
(only showing last batch)
docker logs docker-nginx-1
shows a ton of stuff, but no errors that I can see
docker compose down says no configuration file docker compose up says no configuration file
So I did
sudo systemctl restart docker
chris@FORGE:~/ai/websearch/searxng$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
643a7e4133a2 nginx:latest "sh -c 'cp /docker-e…" 10 days ago Up 14 seconds 0.0.0.0:443->443/tcp, :::443->443/tcp, 0.0.0.0:8088->80/tcp, :::8088->80/tcp docker-nginx-1
2f03f44c2b34 langgenius/dify-api:0.6.15 "/bin/bash /entrypoi…" 10 days ago Up 11 seconds 5001/tcp docker-worker-1
eb017ab09d70 langgenius/dify-api:0.6.15 "/bin/bash /entrypoi…" 10 days ago Up 11 seconds 5001/tcp docker-api-1
d2e51d56de93 postgres:15-alpine "docker-entrypoint.s…" 10 days ago Up 13 seconds (healthy) 5432/tcp docker-db-1
01dd0ec3862c langgenius/dify-sandbox:0.2.1 "/main" 10 days ago Up 13 seconds docker-sandbox-1
c7809d79fca6 redis:6-alpine "docker-entrypoint.s…" 10 days ago Up 14 seconds (health: starting) 6379/tcp docker-redis-1
f965152767be ubuntu/squid:latest "sh -c 'cp /docker-e…" 10 days ago Up 13 seconds 3128/tcp docker-ssrf_proxy-1
57b017c687c1 semitechnologies/weaviate:1.19.0 "/bin/weaviate --hos…" 10 days ago Up 17 seconds docker-weaviate-1
d56cc8b4cbff langgenius/dify-web:0.6.15 "/bin/sh ./entrypoin…" 10 days ago Up 11 seconds 3000/tcp docker-web-1
eea4e237bc5e ghcr.io/open-webui/open-webui:main "bash start.sh" 2 weeks ago Up 14 seconds (health: starting) 0.0.0.0:3000->8080/tcp, :::3000->8080/tcp open-webui
which should have restarted the works.
I don't understand what you mean in #6
docker ps -q | xargs -n 1 docker inspect --format '{{ .Name }}: {{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}'
/docker-nginx-1: 172.22.0.5
/docker-worker-1: 172.22.0.8172.21.0.4
/docker-api-1: 172.22.0.9172.21.0.5
/docker-db-1: 172.22.0.7
/docker-sandbox-1: 172.21.0.2172.24.0.2172.23.0.2
/docker-redis-1: 172.22.0.4
/docker-ssrf_proxy-1: 172.22.0.3172.21.0.3
/docker-weaviate-1: 172.22.0.2
/docker-web-1: 172.22.0.6172.24.0.4
/open-webui: 172.17.0.2172.24.0.3
Your last instruction ' Open the location where you stored the Dify source code, open dify/docker/nginx/conf.d, replace http://api:5001 with http://172.19.0.7:5001, and replace http://web:3000 with http://172.19.0.5:3000. Afterward, restart the Nginx container or reload the configuration.
' can't be followed as dify/docker/ngix/conf.d is a directory. Did you mean default.conf? and are you talking about the proxy_pass?
(This is so frustrating!)
To resolve the 403 error and the issues you're facing with the Docker setup, follow these steps:
Ensure JSON Format is Enabled:
Make sure the settings.yml
file of SearxNG includes the JSON format:
search:
formats:
- html
- json # <-- add this line
Restart the SearxNG container after making this change.
Correct Docker Compose Command:
It seems like the docker-compose.yml
file might not be in the current directory. Ensure you are in the correct directory where the docker-compose.yml
file is located. If the file is missing, you need to create it. Here is a basic structure for the docker-compose.yml
file:
version: '3.8'
services:
api:
image: langgenius/dify-api:0.6.15
ports:
- "5001:5001"
depends_on:
- db
- redis
- weaviate
web:
image: langgenius/dify-web:0.6.15
ports:
- "3000:3000"
depends_on:
- api
worker:
image: langgenius/dify-api:0.6.15
ports:
- "5001:5001"
depends_on:
- db
- redis
- weaviate
db:
image: postgres:15-alpine
environment:
POSTGRES_USER: your_user
POSTGRES_PASSWORD: your_password
POSTGRES_DB: your_db
ports:
- "5432:5432"
redis:
image: redis:6-alpine
ports:
- "6379:6379"
weaviate:
image: semitechnologies/weaviate:1.19.0
ports:
- "8080:8080"
nginx:
image: nginx:latest
ports:
- "80:80"
depends_on:
- web
Nginx Configuration:
Yes, you should edit the default.conf
file in the dify/docker/nginx/conf.d
directory. Update the proxy_pass
directives with the correct IP addresses obtained from the docker inspect
command. For example:
server {
listen 80;
server_name your_domain_or_ip;
location /api/ {
proxy_pass http://172.22.0.9:5001; # Replace with the correct IP
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location / {
proxy_pass http://172.22.0.6:3000; # Replace with the correct IP
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
}
Restart Nginx:
After updating the default.conf
file, restart the Nginx container:
docker restart docker-nginx-1
Check URL Accessibility: Open the URL in your browser to ensure it works:
http://127.0.0.1:8081/search?q=apple&format=json&categories=general
These steps should help resolve the 403 error and ensure proper integration between Dify and SearxNG [1][2].
To continue talking to Dosu, mention @dosu.
Self Checks
Dify version
0.6.15
Cloud or Self Hosted
Self Hosted (Docker)
Steps to reproduce
Followed this tutorial that integrates searxng into dify and ran into problems. https://www.youtube.com/watch?v=yXAJwDtAbLo&ab_channel=MattWilliams Issue looks the same as https://github.com/open-webui/open-webui/issues/2824 I'm not able to connect to the searxng with dify although I have no problems using a browser. I've tried various urls http://192.168.1.15:8080 says I don't have permissions, it is either read protected or or not readable by the server.
ALSO WHY IS THE URL BEING HIDDEN, and WHY DO THE ERROR MESSAGES ONLY LAST A 1/4 of a second. This is very hard to use when things are going wrong.
✔️ Expected Behavior
It to work
❌ Actual Behavior
It didn't work.