[Bug] Failed to download the file

smartyhero commented 1 month ago

Bug Description

I uploaded a file using /file, and then clicked the file to jump to the browser, but the browser returned the following content. I am using the self-hosted appflowy

How to Reproduce

Using the self-hosted appflowy, upload the file and click on the file

Expected Behavior

Can download files normally

Operating System

mac

AppFlowy Version(s)

0.6.7

Screenshots

No response

Additional Context

No response

khorshuheng commented 1 month ago

Can you share your nginx and Appflowy configuration?

smartyhero commented 1 month ago

I did not use the official docker-compose, and I did not use Nginx. I used envoy. The following is my envoy configuration

admin:
  address:
    socket_address:
      address: 127.0.0.1
      port_value: 9901

static_resources:
  # https://www.envoyproxy.io/docs/envoy/latest/configuration/security/secret#example-one-static-resource
  secrets:
    - name: smartyhero
      tls_certificate:
        certificate_chain:
          filename: "/usr/local/appflowy/ssl/smartyhero.pem"
        private_key:
          filename: "/usr/local/appflowy/ssl/smartyhero-key.pem"
        watched_directory:
          path: /usr/local/appflowy/ssl/
  listeners:
    - name: http
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 80
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_http
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: backend
                      domains:
                      - "*"
                      routes:
                        - match:
                            prefix: "/"
                          redirect:
                            https_redirect: true
                http_filters:
                  - name: envoy.filters.http.router
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
    - name: https
      address:
        socket_address:
          address: 0.0.0.0
          port_value: 443
      filter_chains:
        - filters:
            - name: envoy.filters.network.http_connection_manager
              typed_config:
                "@type": type.googleapis.com/envoy.extensions.filters.network.http_connection_manager.v3.HttpConnectionManager
                stat_prefix: ingress_ws_to_ws
                codec_type: AUTO
                use_remote_address: true # 开始x-forwarded-for header skip_xff_append必须为false
                upgrade_configs:
                  - upgrade_type: websocket
                skip_xff_append: false
                access_log:
                  - name: envoy.access_loggers.file
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.access_loggers.file.v3.FileAccessLog
                      path: /usr/local/appflowy/logs/envoy/access.json # 修改日志文件路径
                      json_format:
                        start_time: "%START_TIME%"
                        method: "%REQ(:METHOD)%"
                        path: "%REQ(X-ENVOY-original-path?:PATH)%"
                        protocol: "%PROTOCOL%"
                        user_agent: "%REQ(USER-AGENT)%"
                        request_id: "%REQ(X-REQUEST-ID)%"
                        response_code: "%RESPONSE_CODE%"
                        response_flags: "%RESPONSE_FLAGS%"
                        bytes_received: "%BYTES_RECEIVED%"
                        bytes_sent: "%BYTES_SENT%"
                        duration: "%DURATION%"
                        upstream_service_time: "%RESP(X-ENVOY-UPSTREAM-SERVICE-TIME)%"
                        upstream_host: "%UPSTREAM_HOST%"
                        upstream_cluster: "%UPSTREAM_CLUSTER%"
                        host: "%REQ(:AUTHORITY)%"
                route_config:
                  name: local_route
                  virtual_hosts:
                    - name: vaultwarden
                      domains:
                        - vault.smartyhero.com
                      routes:
                        - match:
                            safe_regex:
                              regex: "^/(.*)$"
                          route:
                            cluster: vaultwarden
                            regex_rewrite:
                              pattern:
                                regex: "^/(.*)$"
                              substitution: /\1
                          request_headers_to_add:
                            - header:
                                key: X-Real-IP
                                value: "%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%"
                              append_action: ADD_IF_ABSENT
                            - header:
                                key: X-Scheme
                                value: "%PROTOCOL%"
                              append_action: ADD_IF_ABSENT
                    - name: share
                      domains:
                        - "dl.smartyhero.com"
                      routes:
                        - match:
                            prefix: /
                          route:
                            cluster: demo
                          request_headers_to_add:
                            - header:
                                key: X-Real-IP
                                value: "%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%"
                              append_action: ADD_IF_ABSENT
                            - header:
                                key: X-Scheme
                                value: "%PROTOCOL%"
                              append_action: ADD_IF_ABSENT
                    - name: gotrue
                      domains:
                        - appflowy.smartyhero.com
                        - appflowy.smartyhero.com:443
                      cors:
                        allow_origin_string_match:
                          - safe_regex:
                              regex: "^appflowy.smartyhero.com$"
                        allow_methods: "GET, POST, OPTIONS, PUT, DELETE, PATCH"
                        allow_headers: "Content-Type, Authorization, Accept, Client-Version"
                        max_age: "3600"
                      routes:
                        - match:
                            safe_regex:
                              regex: "^/gotrue/(.*)$"
                          route:
                            cluster: gotrue
                            regex_rewrite:
                              pattern:
                                regex: "^/gotrue/(.*)$"
                              substitution: /\1
                          request_headers_to_add:
                            - header:
                                key: X-Real-IP
                                value: "%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%"
                              append_action: ADD_IF_ABSENT
                            - header:
                                key: X-Scheme
                                value: "%PROTOCOL%"
                              append_action: ADD_IF_ABSENT
                        - match:
                            safe_regex:
                              regex: "^/minio/(.*)$"
                          route:
                            cluster: minio
                            regex_rewrite:
                              pattern:
                                regex: "^/minio/(.*)$"
                              substitution: /\1
                          request_headers_to_add:
                            - header:
                                key: X-Real-IP
                                value: "%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%"
                              append_action: ADD_IF_ABSENT
                            - header:
                                key: X-Scheme
                                value: "%PROTOCOL%"
                              append_action: ADD_IF_ABSENT
                        - match:
                            safe_regex:
                              regex: "^/(api|ws)(.*)$"
                          route:
                            cluster: appflowy-cloud
                          request_headers_to_add:
                            - header:
                                key: X-Real-IP
                                value: "%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%"
                              append_action: ADD_IF_ABSENT
                            - header:
                                key: X-Scheme
                                value: "%PROTOCOL%"
                              append_action: ADD_IF_ABSENT
                        - match:
                            path: /cat
                          route:
                            cluster: demo
                          request_headers_to_add:
                            - header:
                                key: X-Real-IP
                                value: "%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%"
                              append_action: ADD_IF_ABSENT
                            - header:
                                key: X-Scheme
                                value: "%PROTOCOL%"
                              append_action: ADD_IF_ABSENT
                        - match:
                            safe_regex:
                              regex: "^/(web|assets|favicon.ico).*$"
                          route:
                            cluster: appflowy-cloud-fe
                          request_headers_to_add:
                            - header:
                                key: X-Real-IP
                                value: "%DOWNSTREAM_REMOTE_ADDRESS_WITHOUT_PORT%"
                              append_action: ADD_IF_ABSENT
                            - header:
                                key: X-Scheme
                                value: "%PROTOCOL%"
                              append_action: ADD_IF_ABSENT
                http_filters:
                  - name: envoy.filters.http.router
                    typed_config:
                      "@type": type.googleapis.com/envoy.extensions.filters.http.router.v3.Router
          transport_socket:
            name: envoy.transport_sockets.tls
            typed_config:
              "@type": type.googleapis.com/envoy.extensions.transport_sockets.tls.v3.DownstreamTlsContext
              require_client_certificate: false # 验证客户端证书 mTls
              common_tls_context:
                tls_params:
                  tls_minimum_protocol_version: TLSv1_1 # tls最小的版本
                alpn_protocols: # 支持的http协议版本, 这里要吐槽, 默认不支持http2
                  - h2
                  - http/1.1
                tls_certificate_sds_secret_configs:
                - name: smartyhero
  clusters:
    - name: gotrue
      connect_timeout: 3s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: gotrue
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 127.0.0.1
                      port_value: 8081
    - name: appflowy-cloud
      connect_timeout: 3s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: appflowy-cloud
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 127.0.0.1
                      port_value: 8000
    - name: minio
      connect_timeout: 3s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: minio
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 127.0.0.1
                      port_value: 9002
    - name: appflowy-cloud-fe
      connect_timeout: 3s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: appflowy-cloud-fe
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 127.0.0.1
                      port_value: 3000
    - name: demo
      connect_timeout: 3s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: demo
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 127.0.0.1
                      port_value: 8080
    - name: vaultwarden
      connect_timeout: 3s
      type: STATIC
      lb_policy: ROUND_ROBIN
      load_assignment:
        cluster_name: vaultwarden
        endpoints:
          - lb_endpoints:
              - endpoint:
                  address:
                    socket_address:
                      address: 127.0.0.1
                      port_value: 8899

smartyhero commented 1 month ago

The following is my appflowy configuration. I replaced some confidential information with xxx.

# GoTrue URL that the appflowy service will use to connect to gotrue
# In docker environment, `gotrue` is the hostname of the gotrue servic
GOTRUE_URL=http://127.0.0.1:8081
REDIS_URL=redis://127.0.0.1:6379
GOTRUE_DB_DRIVER=postgres
GOTRUE_SITE_URL=appflowy-flutter://login-callback
URI_ALLOW_LIST=*
APPFLOWY_DATABASE_NAME=appflowy
APPFLOWY_GOTRUE_BASE_URL=http://127.0.0.1:8081
APPFLOWY_DATABASE_URL=postgres://postgres:xxx@127.0.0.1:5432/appflowy
DATABASE_URL=postgres://supabase_auth_admin:root@127.0.0.1:5432/appflowy
GOTRUE_DATABASE_URL=postgres://supabase_auth_admin:root@127.0.0.1:5432/appflowy
#?searchpath=auth

GOTRUE_MAILER_URLPATHS_CONFIRMATION=/gotrue/verify
GOTRUE_MAILER_URLPATHS_INVITE=/gotrue/verify
GOTRUE_MAILER_URLPATHS_RECOVERY=/gotrue/verify
GOTRUE_MAILER_URLPATHS_EMAIL_CHANGE=/gotrue/verify

APPFLOWY_ENVIRONMENT=production
APPFLOWY_REDIS_URI=redis://127.0.0.1:6379

# This file is a template for docker compose deployment
# Copy this file to .env and change the values as needed

# authentication key, change this and keep the key safe and secret
# self defined key, you can use any string
GOTRUE_JWT_SECRET=xxx
APPFLOWY_GOTRUE_JWT_SECRET=xxx
GOTRUE_JWT_EXP=72000
ADMIN_FRONTEND_REDIS_URL=redis://127.0.0.1:6379
ADMIN_FRONTEND_GOTRUE_URL=http://127.0.0.1:8081

# User sign up will automatically be confirmed if this is set to true.
# If you have OAuth2 set up or smtp configured, you can set this to false
# to enforce email confirmation or OAuth2 login instead.
# If you set this to false, you need to either set up SMTP
GOTRUE_MAILER_AUTOCONFIRM=true

# if you enable mail confirmation, you need to set the SMTP configuration below
GOTRUE_SMTP_HOST=smtp.gmail.com
GOTRUE_SMTP_PORT=465
GOTRUE_SMTP_USER=email_sender@some_company.com
GOTRUE_SMTP_PASS=email_sender_password
GOTRUE_SMTP_ADMIN_EMAIL=comp_admin@some_company.com

# gotrue admin
GOTRUE_ADMIN_EMAIL=admin@example.com
GOTRUE_ADMIN_PASSWORD=xxxx

# clicking on email verification link will redirect to this host
# change this to your own domain where you host the docker-compose or gotrue
API_EXTERNAL_URL=https://appflowy.smartyhero.com/

# In docker environment, `postgres` is the hostname of the postgres service
# GoTrue connect to postgres using this url
#GOTRUE_DATABASE_URL=postgres://postgres:85xIz4dPDrfHtakc@127.0.0.1:5432/appflowy?searchpath=auth

# Refer to this for details: https://github.com/AppFlowy-IO/AppFlowy-Cloud/blob/main/doc/AUTHENTICATION.md
# Google OAuth2
GOTRUE_EXTERNAL_GOOGLE_ENABLED=false
GOTRUE_EXTERNAL_GOOGLE_CLIENT_ID=
GOTRUE_EXTERNAL_GOOGLE_SECRET=
GOTRUE_EXTERNAL_GOOGLE_REDIRECT_URI=http://your-host/gotrue/callback
# GitHub OAuth2
GOTRUE_EXTERNAL_GITHUB_ENABLED=true
GOTRUE_EXTERNAL_GITHUB_CLIENT_ID=xxx
GOTRUE_EXTERNAL_GITHUB_SECRET=xxx
GOTRUE_EXTERNAL_GITHUB_REDIRECT_URI=https://appflowy.smartyhero.com/gotrue/callback
# Discord OAuth2
GOTRUE_EXTERNAL_DISCORD_ENABLED=false
GOTRUE_EXTERNAL_DISCORD_CLIENT_ID=
GOTRUE_EXTERNAL_DISCORD_SECRET=
GOTRUE_EXTERNAL_DISCORD_REDIRECT_URI=http://your-host/gotrue/callback

# File Storage
APPFLOWY_S3_USE_MINIO=true
APPFLOWY_S3_MINIO_URL=http://127.0.0.1:9000
# change this if you are using a different address for minio
APPFLOWY_S3_ACCESS_KEY=admin
APPFLOWY_S3_SECRET_KEY=xxx
APPFLOWY_S3_BUCKET=appflowy
APPFLOWY_S3_REGION=us-east-1
GOTRUE_RATE_LIMIT_EMAIL_SENT=100
RUST_LOG=debug
GOTRUE_LOG_LEVEL="debug"
ADMIN_FRONTEND_APPFLOWY_CLOUD_GATEWAY_URL=http://localhost:8000
APPFLOWY_ACCESS_CONTROL=true

# AppFlowy History
APPFLOWY_HISTORY_URL=http://127.0.0.1:50051
APPFLOWY_HISTORY_REDIS_URL=redis://127.0.0.1:6379
APPFLOWY_HISTORY_DATABASE_URL=postgres://postgres:xxx@127.0.0.1:5432/appflowy
APPFLOWY_HISTORY_DATABASE_NAME=appflowy

khorshuheng commented 1 month ago

How are you running the Appflowy cloud services (gotrue, cloud, Redis, postgres etc)? Docker compose with host network mode? Or, did you compile Appflowy cloud and run it using the binary directly on localhost?

As for envoy, I am assuming that you are running the envoy pre built binary, instead of docker?

smartyhero commented 1 month ago

I manually compiled the Appflowy-cloud code on the host, including appflowy-cloud, admin, and gotrue, and ran the relevant binary files directly on the host. I compiled it according to the relevant dockerfile in the appflowy-cloud code base

I used systemd to manage it

envoy is the binary file I downloaded from its github release page

By the way, I am using Ubuntu 22.04 operating system

smartyhero commented 1 month ago

My English is not good, I used Google Translate, please forgive me

khorshuheng commented 1 month ago

Noted, do give me some time to try to replicate your setup. Will update here once I am done.

smartyhero commented 1 month ago

Does the appflowy-cloud deployed using docker-compose have this problem?

khorshuheng commented 1 month ago

This issue is not caused by not using docker-compose. It's certainly possible to use the compiled binaries + systemd and envoy, as long as your configuration on envoy (and the .env files) is correct. The reason I need to replicate your setup, is to determine the envoy config that is equivalent to what we have on nginx.

khorshuheng commented 1 month ago

I was able to replicate the issue - the file downloading issue is not due to the envoy setup etc. Could you try setting APPFLOWY_ACCESS_CONTROL=false , and let us know whether you are able to download the file?

smartyhero commented 1 month ago

I added the environment variable APPFLOWY_ACCESS_CONTROL=false, but it still doesn't work. I re-uploaded the new file and downloaded it again, but it still doesn't work.

khorshuheng commented 1 month ago

Do you mean, you still see the same error message: can't find user uuid from requests ?

And just to be sure: from what we can see on your uploaded environment variables, the APPFLOWY_ACCESS_CONTROL is already set. So instead of adding this variable, we will need to modify the value of the existing variable.

smartyhero commented 1 month ago

I didn't notice that APPFLOWY_ACCESS_CONTROL had been set before, I retested it and now I set APPFLOWY_ACCESS_CONTROL to false and I can download the file

khorshuheng commented 1 month ago

Setting APPFLOWY_ACCESS_CONTROL to false is a workaround, we will see if we can resolve this issue on the server end.

smartyhero commented 1 month ago

OK, I'll set it like this temporarily, thank you for your contribution

AppFlowy-IO / AppFlowy