File Upload are not working?

[tobiashochguertel]

I have a self-hosted server instance of standardnotes, and a self-hosted professional license. I set up the env variable PUBLIC_FILES_SERVER_URL in the .env file. The Standard Notes Desktop App (Mac M1) is logged into my self-hosted Standard Notes Server via the Custom sync server url. My Self-hosted instance is behind a reverse proxy available, the ports for the Server and the Files Server have their own unique DNS names.

Server: https://standardnotes.vps4.hochguertel.work/ Files Server: https://files-standardnotes.vps4.hochguertel.work/

When I try to upload a File, Image etc. I received a Error Message: Unable to start upload session, followed by the message: There was an error while uploading the file.

Can someone help me to get the file upload working? Is there a Debug Mode for the Desktop app, to see a more detailed error message?

I use the latest version of standardnotes/server container. Just set up standardnotes today (22. July.2023) so everything is fresh and latest.

Files Logoutput:

❯ tail -f logs/files*.log
==> logs/files.log <==
db:3306 is unavailable yet - waiting for it to start
db:3306 is up. Proceeding to startup.
cache:6379 is up. Proceeding to startup.
{"level":"info","message":"Server started on port 3104","service":"files"}
db:3306 is up. Proceeding to startup.
cache:6379 is up. Proceeding to startup.
{"level":"info","message":"Server started on port 3104","service":"files"}
db:3306 is up. Proceeding to startup.
cache:6379 is up. Proceeding to startup.
{"level":"info","message":"Server started on port 3104","service":"files"}

==> logs/files-worker.log <==
localhost:3101 is unavailable yet - waiting for it to start
localhost:3101 is up. Proceeding to startup.
{"level":"info","message":"Starting worker...","service":"files"}
localhost:3101 is unavailable yet - waiting for it to start
localhost:3101 is up. Proceeding to startup.
{"level":"info","message":"Starting worker...","service":"files"}
{"level":"info","message":"Alive and kicking!","service":"files"}
localhost:3101 is unavailable yet - waiting for it to start
localhost:3101 is up. Proceeding to startup.
{"level":"info","message":"Starting worker...","service":"files"}

[tobiashochguertel]

I researched a bit, and documented it in Discord in a thread. The thread can be found here.

[tobiashochguertel]

My Docker Compose configuration with Traefik as Reverse proxy:

---
version: "3.8"

networks:
  standardnotes_self_hosted:
    name: standardnotes_self_hosted
  proxy:
    external: true
  intranet:
    external: true

services:
  webclient:
    image: standardnotes/web:stable
    env_file: webclient.env
    # ports:
    #   - 3001:3001
    networks:
      - standardnotes_self_hosted
      - proxy
      - intranet
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=proxy"
      # https://standardnotes.vps4.hochguertel.work
      - "traefik.http.routers.standardnotes.entrypoints=http"
      - "traefik.http.routers.standardnotes.rule=Host(`standardnotes.vps4.hochguertel.work`)"
      - "traefik.http.routers.standardnotes.service=standardnotes"
      - "traefik.http.services.standardnotes.loadbalancer.server.port=3001"
      - "traefik.http.middlewares.standardnotes-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.standardnotes.middlewares=standardnotes-https-redirect"
      - "traefik.http.routers.standardnotes-secure.entrypoints=https"
      - "traefik.http.routers.standardnotes-secure.rule=Host(`standardnotes.vps4.hochguertel.work`)"
      - "traefik.http.routers.standardnotes-secure.service=standardnotes"
      - "traefik.http.routers.standardnotes-secure.tls=true"
      - "traefik.http.routers.standardnotes-secure.tls.certresolver=http"
  server:
    image: standardnotes/server
    env_file: .env
    container_name: server_self_hosted
    # ports:
    #   - 3000:3000
    #   - 3125:3104
    volumes:
      - ./logs:/var/lib/server/logs
      - ./uploads:/opt/bundled/files/packages/files/dist/uploads:Z
    networks:
      - standardnotes_self_hosted
      - proxy
      - intranet
    labels:
      - "traefik.enable=true"
      - "traefik.docker.network=proxy"
      # https://api.standardnotes.vps4.hochguertel.work
      - "traefik.http.routers.api-standardnotes.entrypoints=http"
      - "traefik.http.routers.api-standardnotes.rule=Host(`api.standardnotes.vps4.hochguertel.work`)"
      - "traefik.http.routers.api-standardnotes.service=api-standardnotes"
      - "traefik.http.services.api-standardnotes.loadbalancer.server.port=3000"
      - "traefik.http.middlewares.api-standardnotes-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.api-standardnotes.middlewares=standardnotes-https-redirect"
      - "traefik.http.routers.api-standardnotes-secure.entrypoints=https"
      - "traefik.http.routers.api-standardnotes-secure.rule=Host(`api.standardnotes.vps4.hochguertel.work`)"
      - "traefik.http.routers.api-standardnotes-secure.service=api-standardnotes"
      - "traefik.http.routers.api-standardnotes-secure.tls=true"
      - "traefik.http.routers.api-standardnotes-secure.tls.certresolver=http"
      # https://files.standardnotes.vps4.hochguertel.work
      - "traefik.http.routers.files-standardnotes.entrypoints=http"
      - "traefik.http.routers.files-standardnotes.rule=Host(`files.standardnotes.vps4.hochguertel.work`)"
      - "traefik.http.routers.files-standardnotes.service=files-standardnotes"
      - "traefik.http.services.files-standardnotes.loadbalancer.server.port=3104"
      - "traefik.http.middlewares.files-standardnotes-https-redirect.redirectscheme.scheme=https"
      - "traefik.http.routers.files-standardnotes.middlewares=files-standardnotes-https-redirect"
      - "traefik.http.routers.files-standardnotes-secure.entrypoints=https"
      - "traefik.http.routers.files-standardnotes-secure.rule=Host(`files.standardnotes.vps4.hochguertel.work`)"
      - "traefik.http.routers.files-standardnotes-secure.service=files-standardnotes"
      - "traefik.http.routers.files-standardnotes-secure.tls=true"
      - "traefik.http.routers.files-standardnotes-secure.tls.certresolver=http"

  localstack:
    image: localstack/localstack:1.3
    container_name: localstack_self_hosted
    # expose:
    #   - 4566
    restart: unless-stopped
    environment:
      - SERVICES=sns,sqs
      - HOSTNAME_EXTERNAL=localstack
      - LS_LOG=warn
    volumes:
      - ./localstack_bootstrap.sh:/etc/localstack/init/ready.d/localstack_bootstrap.sh
    networks:
      - standardnotes_self_hosted

  db:
    image: mysql:8
    container_name: db_self_hosted
    environment:
      - MYSQL_DATABASE=standard_notes_db
      - MYSQL_USER=std_notes_user
      - MYSQL_ROOT_PASSWORD=SECRET
      - MYSQL_PASSWORD=SECRET
    # expose:
    #   - 3306
    restart: unless-stopped
    command: --default-authentication-plugin=mysql_native_password --character-set-server=utf8mb4 --collation-server=utf8mb4_general_ci
    volumes:
      - ./data/mysql:/var/lib/mysql
      - ./data/import:/docker-entrypoint-initdb.d
    networks:
      - standardnotes_self_hosted

  cache:
    image: redis:6.0-alpine
    container_name: cache_self_hosted
    volumes:
      - ./data/redis/:/data
    # expose:
    #   - 6379
    restart: unless-stopped
    networks:
      - standardnotes_self_hosted

My .env Files:

File: .env

######
# DB #
######

DB_HOST=db
DB_PORT=3306
DB_USERNAME=std_notes_user
DB_PASSWORD=SECRET
DB_DATABASE=standard_notes_db
DB_TYPE=mysql

#########
# CACHE #
#########

REDIS_PORT=6379
REDIS_HOST=cache
CACHE_TYPE=redis

########
# KEYS #
########

AUTH_JWT_SECRET=SECRET
AUTH_SERVER_ENCRYPTION_SERVER_KEY=SECRET
VALET_TOKEN_SECRET=SECRET

PUBLIC_FILES_SERVER_URL=https://files.standardnotes.vps4.hochguertel.work

File: webclient.env

PORT=3001

DEFAULT_SYNC_SERVER=https://api.standardnotes.vps4.hochguertel.work

# Subscription related endpoints
DASHBOARD_URL=http://standardnotes.com/dashboard
PLANS_URL=https://standardnotes.com/plans
PURCHASE_URL=https://standardnotes.com/purchase

# Used by Rails internals and not Standard Notes related
SECRET_KEY_BASE=SECRET

[moughxyz]

@tobiashochguertel does that mean you got it working?

[tobiashochguertel]

@moughxyz Nope, it doesn't work. I thought I add a complete setup here to show how I wired everything together. I still can't upload any image, or file.

[tobiashochguertel]

You can also access my self-hosted platform and try it out yourself, that might help to debug?

API (Server): https://api.standardnotes.vps4.hochguertel.work Files (Server): https://files.standardnotes.vps4.hochguertel.work

Standard Notes Web App: https://standardnotes.vps4.hochguertel.work

I tested it with my Phone's (iOS, Android) and Desktop Clients (Mac M1), the result was the same error message:

I tried it also without traefik, by exposing the port via docker and opening the port in the firewall and setting up the variable PUBLIC_FILES_SERVER_URL to be http://45.85.249.160:3125.

Exposing Port:

Set PUBLIC_FILES_SERVER_URL to server IP with port information:

Open Ports in the firewall (ufw):

Restarting Docker Containers / Compose Service:

Result: Error messages

[tobiashochguertel]

I also tried the latest Web App Version / Development from today:

[moughxyz]

Try monitoring logs in these locations:

• tail -f logs/*.log
• tail -f logs/*.err

[tobiashochguertel]

When I try to upload, I don't see anything in the logs. I tried to upload several files. Here are my log files:

logs.zip

[tobiashochguertel]

Is there a way to test the files server standalone? Via Postman Or curl?

[tobiashochguertel]

What I see in traefik logs is the following:

reverse-proxy_1         | 2.211.164.122 - - [24/Jul/2023:14:35:45 +0000] "POST /v1/files/valet-tokens HTTP/2.0" 403 254 "-" "-" 3472 "api-standardnotes-secure@docker" "http://172.27.0.2:3000" 45ms
reverse-proxy_1         | 2.211.164.122 - - [24/Jul/2023:14:35:48 +0000] "POST /v1/files/valet-tokens HTTP/2.0" 403 254 "-" "-" 3473 "api-standardnotes-secure@docker" "http://172.27.0.2:3000" 51ms

403 Forbidden from the api.standardnotes.vps4.hochguertel.work

[tobiashochguertel]

I figured out that I have to run manual SQL commands to enable the purchased features?

https://standardnotes.com/help/self-hosting/subscriptions

Now it works! Not so well documented for my perspective.

docker compose exec db sh -c "MYSQL_PWD=\$MYSQL_ROOT_PASSWORD mysql \$MYSQL_DATABASE -e \
    'INSERT INTO user_roles (role_uuid , user_uuid) VALUES ((SELECT uuid FROM roles WHERE name=\"PRO_USER\" ORDER BY version DESC limit 1) ,(SELECT uuid FROM users WHERE email=\"tobias.hochguertel@googlemail.com\")) ON DUPLICATE KEY UPDATE role_uuid = VALUES(role_uuid);' \
"

docker compose exec db sh -c "MYSQL_PWD=\$MYSQL_ROOT_PASSWORD mysql \$MYSQL_DATABASE -e \
    'INSERT INTO user_subscriptions SET uuid=UUID(), plan_name=\"PRO_PLAN\", ends_at=8640000000000000, created_at=0, updated_at=0, user_uuid=(SELECT uuid FROM users WHERE email=\"tobias.hochguertel@googlemail.com\"), subscription_id=1, subscription_type=\"regular\";' \
"