search

AppFlowy-IO / AppFlowy-Cloud

AppFlowy is an open-source alternative to Notion. You are in charge of your data and customizations. Built with Flutter and Rust.
GNU Affero General Public License v3.0
982 stars 207 forks source link

Please tell me how to set up nginx as a frontend and connect it to Appflowy-Cloud using a reverse proxy. #717

Closed tocynics closed 1 month ago

tocynics commented 1 month ago

I want to make the appflowy-cloud environment, which is only connected to the bridge in a virtual environment, accessible via the already deployed nginx reverse proxy.

The configuration method is shown in the issue below, but data was not synchronized even after adding this configuration. https://github.com/AppFlowy-IO/AppFlowy-Cloud/issues/240

I was unable to verify data synchronization between different devices. Also, the value of Document Storage did not change even after adding documents. スクリーンショット 2024-08-08 022540

khorshuheng commented 1 month ago

Can you share your nginx configuration?

tocynics commented 1 month ago

Thanks!

Here is the configuration file.

###########################################################
### virtual domain
###########################################################
map $http_host $backend_appflowy {
    default localhost:80;
    {FQDN}   10.21.137.138;
}

server {
    include /etc/nginx/includes/listen_443.conf;

    server_name {FQDN};

    access_log  /var/log/nginx/$server_name/access.log;

    location / {
        proxy_set_header X-Real-IP  $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header X-Forwarded-Port 443;
        proxy_set_header Host $host;
        proxy_buffering off;
        proxy_cache off;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "Upgrade";
        proxy_pass http://$backend_appflowy;
    }
}
khorshuheng commented 1 month ago

Is there any error on the nginx log, specifically, for the websocket related endpoint? (Eg. URL with prefix /ws)

tocynics commented 1 month ago

Thank you for your confirmation. Please find the access log below. As you can see, a 502 error is being returned for the /ws endpoint.

(I apologize if my previous response, which was translated from Japanese by Gemini, came across as rude. Thank you for your understanding.)

{global ip} - - [09/Aug/2024:22:03:44 +0900] "GET /ws/v1 HTTP/1.1" 502 157 "-" "-"
{global ip} - - [09/Aug/2024:22:03:59 +0900] "GET /ws/v1 HTTP/1.1" 502 157 "-" "-"
tocynics commented 1 month ago

To reduce the nginx configuration to the most basic settings, I adjusted the configuration file as shown below. Unfortunately, this did not resolve the issue.

###########################################################
### virtual domain
###########################################################
map $http_host $backend_appflowy {
    default localhost:80;
    {FQDN}   10.21.137.138:80;
}

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

server {
    server_name {FQDN};

    include /etc/nginx/includes/listen_443.conf;
    include /etc/nginx/includes/ssl_{FQDN}.conf;

    access_log  /var/log/nginx/$server_name/access.log;

    proxy_set_header Host $host;
    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

    location / {
        proxy_pass http://$backend_appflowy;
    }
}
khorshuheng commented 1 month ago

By default, the nginx service in the docker compose file attempts to map to port 80 on the host: https://github.com/AppFlowy-IO/AppFlowy-Cloud/blob/main/docker-compose.yml#L9

Since you already have nginx installed on your host, which, in all likelihood, is already listening on 80, when you run docker compose, the nginx service (within docker compose) would have failed. The error message will look something like: 0.0.0.0:80: bind: address already in use.

Hence, for this to work, you will have to modify the docker compose file to map the nginx port to a different port, say, 8080. Then, in your nginx config:

    location / {
        proxy_pass http://localhost:8080;
    }
tocynics commented 1 month ago

I apologize for the insufficient explanation.

Below is a detailed breakdown of the server configuration for nginx and Appflowy-Cloud:

Host Server

LXC Container: nginx

LXC Container: Appflowy-Cloud

Client PC 01

Client PC 02

The frontend nginx is configured separately from the Appflowy-Cloud server, so there is no port conflict.

Additionally, to compare the behavior of the nginx reverse proxy, I've conducted tests by directly accessing the Appflowy-Cloud server. As a connection method, I added an entry to the hosts file of Client PCs 01 and 02 and confirmed that document synchronization was successful.

# C:\Windows\System32\drivers\etc\hosts
10.0.0.138 {FQDN}
khorshuheng commented 1 month ago

Ah, I see. I have wrongly assumed that the docker compose setup and nginx are on the same machine. Allow me some time to replicate your setup and try to reproduce the issue, as I am not too familiar with LXC.

How are the different services (eg nginx, Appflowy cloud, gotrue, admin frontend, postgres) running within the Appflowy Cloud LXC container? Are you running them using docker compose? Or, are these services on different LXC containers?

In addition, are you currently using HTTP or HTTPS?

tocynics commented 1 month ago

Of course! Appflowy Cloud is running on an LXC container. It is started using docker compose on the LXC container.

# entering lxc container
lxc exec appflowy bash

# docker version
root@appflowy:~# docker -v
Docker version 27.1.1, build 6312585

# entering App
root@appflowy:~# cd /opt/AppFlowy-Cloud/

# docker compose up
root@appflowy:~# docker compose up -d
...
[+] Running 12/12
 ✔ Network appflowy-cloud_default               Created
 ✔ Volume "appflowy-cloud_postgres_data"        Created
 ✔ Volume "appflowy-cloud_minio_data"           Created
 ✔ Container appflowy-cloud-appflowy_cloud-1    Started
 ✔ Container appflowy-cloud-gotrue-1            Started
 ✔ Container appflowy-cloud-ai-1                Started
 ✔ Container appflowy-cloud-admin_frontend-1    Started
 ✔ Container appflowy-cloud-nginx-1             Started
 ✔ Container appflowy-cloud-appflowy_history-1  Started
 ✔ Container appflowy-cloud-redis-1             Started
 ✔ Container appflowy-cloud-minio-1             Started
 ✔ Container appflowy-cloud-postgres-1          Started

# docker ps
root@appflowy:/opt/AppFlowy-Cloud# docker ps -a
CONTAINER ID   IMAGE                                 COMMAND                  CREATED        STATUS                          PORTS                                                                      NAMES
e1b890b489f3   minio/minio                           "/usr/bin/docker-ent…"   15 hours ago   Up 15 hours                     9000/tcp                                                                   appflowy-cloud-minio-1
02488a1f5495   redis                                 "docker-entrypoint.s…"   15 hours ago   Up 15 hours                     6379/tcp                                                                   appflowy-cloud-redis-1
99e2c9d57dee   appflowy-cloud-postgres               "docker-entrypoint.s…"   15 hours ago   Up 15 hours                     5432/tcp                                                                   appflowy-cloud-postgres-1
c9babe5a8a4a   appflowyinc/appflowy_ai:latest        "sh -c 'uvicorn main…"   15 hours ago   Restarting (1) 10 seconds ago                                                                              appflowy-cloud-ai-1
1ed80e1ad6e8   appflowyinc/gotrue:latest             "./gotrue"               15 hours ago   Up 15 hours                                                                                                appflowy-cloud-gotrue-1
38df6121db98   nginx                                 "/docker-entrypoint.…"   15 hours ago   Up 15 hours                     0.0.0.0:80->80/tcp, :::80->80/tcp, 0.0.0.0:443->443/tcp, :::443->443/tcp   appflowy-cloud-nginx-1
d540647ff854   appflowyinc/appflowy_history:latest   "appflowy_history"       15 hours ago   Up 15 hours                                                                                                appflowy-cloud-appflowy_history-1
e354b858d068   appflowyinc/appflowy_cloud:latest     "appflowy_cloud"         15 hours ago   Up 15 hours                                                                                                appflowy-cloud-appflowy_cloud-1
191ade9271c7   appflowyinc/admin_frontend:latest     "admin_frontend"         15 hours ago   Up 15 hours                                                                                                appflowy-cloud-admin_frontend-1

We are connecting to nginx via HTTPS. Additionally, since we confirmed direct connection to Appflowy Cloud, we have modified the nginx configuration accordingly.

###########################################################
### virtual domain
###########################################################
map $http_host $backend_appflowy {
    default localhost:80;
    {FQDN}   10.21.137.138:443;
}

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

server {
    server_name {FQDN};

    include /etc/nginx/includes/listen_443.conf;
    include /etc/nginx/includes/ssl_{FQDN}.conf;

    access_log  /var/log/nginx/$server_name/access.log;

    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

    location / {
        proxy_pass https://$backend_appflowy;
    }
}

I've modified the nginx configuration as mentioned above, but I'm still getting the 502 error as before.

{global ip} - - [10/Aug/2024:16:54:52 +0900] "GET /ws/v1 HTTP/1.1" 502 157 "-" "-"
{global ip} - - [10/Aug/2024:16:55:07 +0900] "GET /ws/v1 HTTP/1.1" 502 157 "-" "-"
{global ip} - - [10/Aug/2024:16:55:22 +0900] "GET /ws/v1 HTTP/1.1" 502 157 "-" "-"
{global ip} - - [10/Aug/2024:16:55:37 +0900] "GET /ws/v1 HTTP/1.1" 502 157 "-" "-"
{global ip} - - [10/Aug/2024:16:55:52 +0900] "GET /ws/v1 HTTP/1.1" 502 157 "-" "-"
khorshuheng commented 1 month ago

Thanks for the additional info. When you are inside the nginx LXC container, are you able to ping the IP for Appflowy cloud LXC? Also, does this work when you are using only http and not https?

tocynics commented 1 month ago

Yes! Two-way ping is possible.

proxy(nginx) LXC container

proxy LXC container to appflowy LXC container

root@proxy:~# hostname
proxy
root@proxy:~# hostname -I
10.0.0.120 10.21.137.120 192.168.0.120 240d:1e:42c:9000::120
root@proxy:~# ping 10.21.137.138
PING 10.21.137.138 (10.21.137.138) 56(84) bytes of data.
64 bytes from 10.21.137.138: icmp_seq=1 ttl=64 time=0.036 ms
64 bytes from 10.21.137.138: icmp_seq=2 ttl=64 time=0.056 ms
^C
--- 10.21.137.138 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1016ms
rtt min/avg/max/mdev = 0.036/0.046/0.056/0.010 ms

appflowy LXC container

appflowy LXC container to proxy LXC container

root@appflowy:~# hostname
appflowy
root@appflowy:~# hostname -I
10.0.0.138 172.17.0.1 172.18.0.1 10.21.137.138 240d:1e:42c:9000:216:3eff:fefc:8c46
root@appflowy:~# ping 10.21.137.120
PING 10.21.137.120 (10.21.137.120) 56(84) bytes of data.
64 bytes from 10.21.137.120: icmp_seq=1 ttl=64 time=0.031 ms
64 bytes from 10.21.137.120: icmp_seq=2 ttl=64 time=0.048 ms
^C
--- 10.21.137.120 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1048ms
rtt min/avg/max/mdev = 0.031/0.039/0.048/0.008 ms

As additional information, while document synchronization between different devices was unsuccessful, workspace names were synchronized. For instance, when the workspace name was changed from "Main" to "Main01" on "Client PC 01," the same change was reflected on "Client PC 02".

tocynics commented 1 month ago

I will reconfigure nginx to use HTTP for both the frontend and reverse proxy connections, and then get back to you!

tocynics commented 1 month ago

I've changed the configuration to use HTTP. But, I didn't realize that Google OAuth couldn't use HTTP. I'd like to try configuring it in a different way.

google cloud api

スクリーンショット 2024-08-10 201032

nginx: appflowy.conf

###########################################################
### virtual domain
###########################################################
map $http_host $backend_appflowy {
    default localhost:80;
    {FQDN}   10.21.137.138:80;
}

map $http_upgrade $connection_upgrade {
    default upgrade;
    ''      close;
}

server {
    server_name {FQDN};

    include /etc/nginx/includes/listen_443.conf;
    include /etc/nginx/includes/ssl_{FQDN}.conf;

    access_log  /var/log/nginx/$server_name/access.log;

    proxy_http_version 1.1;
    proxy_set_header Upgrade $http_upgrade;
    proxy_set_header Connection $connection_upgrade;

    location / {
        proxy_pass http://$backend_appflowy;
    }
}
systemctl restart nginx

/opt/Appflowy-Cloud/.env

スクリーンショット 2024-08-10 201248

docker compose down
...

docker compose up -d

Appflowy Error

スクリーンショット 2024-08-10 202253

tocynics commented 1 month ago

I found a miss in the Nginx configuration. Adding listen 80; allowed me to verify HTTP connections. Unfortunately, the issue persisted, and I continued to see 502 errors in the Nginx logs. Document synchronization between devices was still unsuccessful. スクリーンショット 2024-08-10 211556

khorshuheng commented 1 month ago

Thanks for your effort. I will try to replicate this, will reply again in a day or two.

tocynics commented 1 month ago

The pleasure is all mine! Thank you very much as well! Please don't hesitate to ask if you need any more information or assistance with LXC configuration and anything else.

khorshuheng commented 1 month ago

Can you try replacing:

proxy_pass http://$backend_appflowy;

with

proxy_pass http://10.21.137.138:80;
tocynics commented 1 month ago

I see! It has been synchronized. It seems I made a mistake in the settings. Thank you very much for your help!

tocynics commented 1 month ago

I have deleted the post that contained a mistake.