Unknown error without any info/log to troubleshoot

[arkanoid87]

Do you want to request a feature or report a bug? bug

What is the current behavior? Unknown error without any info

If the current behavior is a bug, please provide the steps to reproduce and if possible a minimal demo of the problem. client > TLS terminator reverse proxy (traefik) > nextcloud (+ app) + documentserver using traefik:latest, nextcloud:latest and onlyoffice/documentserver:latest in a docker-compose stack. Stacks seems correctly configured: I successfully set all the urls (even http internal ones) in nextcloud app config and both nextcloud and documentserver public https endpoints are reachable. Problem: open ANY document (tested newly created one and the xlsx you provide as example), tab opens, onlyoffice seems to load, then a misterious "unknown error" popup.

What is the expected behavior? Documents opens correctly in new tab

Did this work in previous versions of DocumentServer? Tested 5.3.0.243, same problem there

DocumentServer version: 5.4.1.39

Operating System: Ubuntu server 18.04

Browser version: Firefox 69.0.1

I've been searching for hints on how to debug the problem for hours unsuccessfully. Been loooking into:

• browser js console
• traefik logs
• nextcloud logs
• documentserver logs

Also tried:

• disable CSP
• tail -f /var/log/onlyoffice/documentserver/*/.log (no output here when triggering the problem)
• tail -f /var/www/html/data/nextcloud.log (no ouput here when triggering the problem)

More info:

• browser dev tools shows no network errors (code 200 everywhere)

It's issue ONLYOFFICE/DocumentServer#666 for a reason

[arkanoid87]

By manually analysing the js stacktrace I ended up nearby this failed XMLHttpRequest

"wss://office.mydomain.com/cache/files/4009520959/Editor.bin/Editor.bin?md5=vyDJoVrnjfh3o02BppqNsA&expires=1573258415&disposition=attachment&ooname=output.bin"

EDIT: testing websocket connection manually from remote host with websocat tool returns valid connection:

websocat -vvv wss://office.mydomain.com/5.4.1-39//doc/3555953290/c/259/vtqjoun2/websocket ... [INFO websocat::ws_client_peer] Connected to ws [DEBUG websocat::ws_peer] incoming text ... ["{\"type\":\"license\",\"license\":{\"type\":3,\"light\":false,\"mode\":0,\"rights\":1,\"buildVersion\":\"5.4.1\",\"buildNumber\":39,\"branding\":false,\"customization\":false,\"plugins\":false}}"]

also websocat -vvv wss://office.mydomain.com/5.4.1-39//spellchecker/doc/3555953290/c/497/ebs0u2ik/websocket works correctly

[arkanoid87]

I've tried removing TLS termination on reverse proxy but the problem is still there even with 100% HTTP

I've increased to 'loglevel' => 0 and this is the output of data/nextcloud.log when I trigger the problem

{"reqId":"2GkFSnxWzA3EFNtCoJzt","level":0,"time":"2019-10-10T02:16:49+00:00","remoteAddr":"2.238.151.49","user":"myuser","app":"onlyoffice","method":"GET","url":"\/apps\/onlyoffice\/320?filePath=%2FExample%20Spreadsheet%20Title.xlsx","message":"Open: 320 \/Example Spreadsheet Title.xlsx","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:69.0) Gecko\/20100101 Firefox\/69.0","version":"17.0.0.9"} {"reqId":"p0Km3mT3Fo9uC4Hvm4ZC","level":0,"time":"2019-10-10T02:16:51+00:00","remoteAddr":"2.238.151.49","user":"myuser","app":"onlyoffice","method":"GET","url":"\/apps\/onlyoffice\/ajax\/config\/320?filePath=%2FExample%20Spreadsheet%20Title.xlsx","message":"Config is generated for: 320 with key ocz4c80wietg_320_1570648071","userAgent":"Mozilla\/5.0 (X11; Ubuntu; Linux x86_64; rv:69.0) Gecko\/20100101 Firefox\/69.0","version":"17.0.0.9"} {"reqId":"c3vUnoLQzpWmO4MIz1wx","level":0,"time":"2019-10-10T02:16:53+00:00","remoteAddr":"172.21.0.4","user":"--","app":"onlyoffice","method":"POST","url":"\/apps\/onlyoffice\/track?doc=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJmaWxlSWQiOjMyMCwib3duZXJJZCI6ImphY2siLCJzaGFyZVRva2VuIjpudWxsLCJhY3Rpb24iOiJ0cmFjayJ9.6tqJL30rrmFIxdgbAB0YcDZuDngzZ1OaZheevtY9rgo","message":"Track: 320 status 1","userAgent":"--","version":"17.0.0.9"} {"reqId":"c3vUnoLQzpWmO4MIz1wx","level":0,"time":"2019-10-10T02:16:53+00:00","remoteAddr":"172.21.0.4","user":"--","app":"onlyoffice","method":"POST","url":"\/apps\/onlyoffice\/track?doc=eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJmaWxlSWQiOjMyMCwib3duZXJJZCI6ImphY2siLCJzaGFyZVRva2VuIjpudWxsLCJhY3Rpb24iOiJ0cmFjayJ9.6tqJL30rrmFIxdgbAB0YcDZuDngzZ1OaZheevtY9rgo","message":"Track: 320 status 1 result 0","userAgent":"--","version":"17.0.0.9"}

[raph2i]

Hey guys, same issue here. Traefik 2 - nextcloud-17-apache - documentserver

[alexanderonlyoffice]

Hello @arkanoid87, it looks likes there is some problem with the functioning of the Document Server.

Please check the Document Server by entering the address _https://documentserver_address/healthcheck/_ and send us the response you get in browser.

Enable extending logging for the Document Server: open /etc/onlyoffice/documentserver/log4js/production.json and replace the 'WARN' value with 'DEBUG' of the "level" parameter. Then restart all services of the Document Server with a command supervisorctl restart all. Then open ONLYOFFICE integration app settings page in Nextcloud and click on the 'Save' button. check the logs after that.

[samuel-p]

I have the same issue since I upgraded Traefik to v2. /healthcheck/ returns true.

[derdrave]

Same here with traefik2(.0.4)

i changed logging to debug and did two things: First i went to the settings-page and saved my settings again and second i tried to open a document.. where the referenced error appears.

the nginx-log from location "/var/log/onlyoffice/documentserver/nginx.error.log" shows these errors:

2019/11/04 10:09:05 [error] 710#710: 1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.2, server: , request: "GET /welcome/ HTTP/1.1", upstream: "http://127.0.0.1:8000/welcome/", host: "office.yourserver.com" 2019/11/04 10:09:05 [error] 710#710: 1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.2, server: , request: "GET /welcome/ HTTP/1.1", upstream: "http://127.0.0.1:8000/welcome/", host: "office.yourserver.com" 2019/11/04 10:09:05 [error] 710#710: 1 no live upstreams while connecting to upstream, client: 172.18.0.2, server: , request: "GET /welcome/ HTTP/1.1", upstream: "http://docservice/welcome/", host: "office.yourserver.com" 2019/11/04 10:09:05 [error] 710#710: 1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.2, server: , request: "GET /welcome/ HTTP/1.1", upstream: "http://127.0.0.1:8000/welcome/", host: "office.yourserver.com" 2019/11/04 10:09:05 [error] 710#710: 1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.2, server: , request: "GET /welcome/ HTTP/1.1", upstream: "http://127.0.0.1:8000/welcome/", host: "office.yourserver.com" 2019/11/04 10:09:06 [error] 710#710: 1 no live upstreams while connecting to upstream, client: 172.18.0.2, server: , request: "GET /welcome/ HTTP/1.1", upstream: "http://docservice/welcome/", host: "office.yourserver.com" 2019/11/04 10:09:06 [error] 710#710: 1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.2, server: , request: "GET /welcome/ HTTP/1.1", upstream: "http://127.0.0.1:8000/welcome/", host: "office.yourserver.com" 2019/11/04 10:09:06 [error] 710#710: 1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.2, server: , request: "GET /welcome/ HTTP/1.1", upstream: "http://127.0.0.1:8000/welcome/", host: "office.yourserver.com" 2019/11/04 10:09:06 [error] 710#710: *1 no live upstreams while connecting to upstream, client: 172.18.0.2, server: , request: "GET /welcome/ HTTP/1.1", upstream: "http://docservice/welcome/", host: "office.yourserver.com"

--> client ip (client: 172.18.0.2) is internal ip of my traefik.. i think it should be either the nextcloud-ip or my real-client-ip?

when i set up onlyoffice and nextcloud dockers with an haproxy docker (official) i have no problems with my setups..

[derdrave]

i found a temporary solution by reconfiguring nginx inside the docker:

in file: /etc/nginx/includes/http-common.conf i commented out this line (bottom of the file):

proxy_set_header Connection $proxy_connection; 
-->
# proxy_set_header Connection $proxy_connection; 

and then i did: service nginx restart

Now i can open onlyoffice-documents from nextcloud... but I'm still not sure if this bug is on traefik or onlyoffice side..

edited on 2019/11/29: service nginx restart .. but i think everyone did realize my mistake...

[senguendk]

i found a temporary solution by reconfiguring nginx inside the docker:

in file: /etc/nginx/includes/http-common.conf i commented out this line (bottom of the file):

proxy_set_header Connection $proxy_connection; 
-->
# proxy_set_header Connection $proxy_connection; 

and then i did: service restart nginx

Now i can open onlyoffice-documents from nextcloud... but I'm still not sure if this bug is on traefik or onlyoffice side..

I can confirm that this workaround works.

[Maurotb]

I have the same issue since I upgraded Traefik to v2. /healthcheck/ returns true.

Same problem when update to traefik 2 Workarround # proxy_set_header Connection $proxy_connection; work for me

[LinneyS]

See the healthcheck status. When setting up the proxy, you must transfer the protocol, host and port to the documentserver. Look for errors while saving connection settings. Enable full docuemtnserver and nextcloud logs and see error messages.

@arkanoid87 Could you please specify if the problem recurs?

[calvinbui]

For those on Traefik 2, I was able to fix the problem with these labels/headers:

traefik.http.routers.onlyoffice-secure.entrypoints: "web-secure"
traefik.http.routers.collabora-secure.tls: "true"
traefik.http.routers.collabora-secure.tls.certresolver: letsencrypt
traefik.http.services.onlyoffice.loadbalancer.server.port: "80"
traefik.http.routers.onlyoffice-secure.middlewares: "onlyoffice-sslheaders"
traefik.http.middlewares.onlyoffice-sslheaders.headers.customrequestheaders.X-Forwarded-Proto: "https"

Similarly if you're having CORS problems, use the CORS header as well

traefik.http.routers.onlyoffice-secure.middlewares: "onlyoffice-sslheaders, onlyoffice-cors"
traefik.http.middlewares.onlyoffice-sslheaders.headers.customrequestheaders.X-Forwarded-Proto: "https"
traefik.http.middlewares.onlyoffice-cors.headers.accessControlAllowOrigin: "*"

[wcdgit]

Similarly if you're having CORS problems, use the CORS header as well

traefik.http.routers.onlyoffice-secure.middlewares: "onlyoffice-sslheaders, onlyoffice-cors"
traefik.http.middlewares.onlyoffice-sslheaders.headers.customrequestheaders.X-Forwarded-Proto: "https"
traefik.http.middlewares.onlyoffice-cors.headers.accessControlAllowOrigin: "*"

That's working for me, too. Thank you so much!

My docker-compose config:

version: '3'

services:
  onlyoffice:
    container_name: onlyoffice
    image: onlyoffice/documentserver:latest
    restart: unless-stopped
    stdin_open: true
    tty: true
    volumes:
      - ${LOCAL_CONF_DIR}/onlyoffice:/var/log/onlyoffice
    environment:
      JWT_ENABLED: "true"
      JWT_SECRET: ${SECRET}
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.${HOSTNAME}-http.entrypoints=web"
      - "traefik.http.routers.${HOSTNAME}-http.rule=Host(`${HOSTNAME}.${DOMAIN0}`)"
      - "traefik.http.routers.${HOSTNAME}-http.middlewares=https_redirect@file"
      - "traefik.http.routers.${HOSTNAME}-https.entrypoints=websecure"
      - "traefik.http.routers.${HOSTNAME}-https.rule=Host(`${HOSTNAME}.${DOMAIN0}`)"
      - "traefik.http.routers.${HOSTNAME}-https.tls=true"
      - "traefik.http.routers.${HOSTNAME}-https.middlewares=sts@file,onlyoffice-headers"
      - "traefik.http.services.${HOSTNAME}.loadbalancer.server.port=80"

      ## Middleware definition
      # Headers for onlyoffice, https://github.com/ONLYOFFICE/onlyoffice-nextcloud/issues/151
      - "traefik.http.middlewares.onlyoffice-headers.headers.customrequestheaders.X-Forwarded-Proto=https"
      - "traefik.http.middlewares.onlyoffice-headers.headers.accessControlAllowOrigin=*"

networks:
  default:
    external:
      name: ${NETWORK}

[derdrave]

does work for me too. thanks to @calvinbui and also to @wcdgit for the full docker-compose for checkup-purposes!

[leonfaeth]

@calvinbui 's labels fixed it for me, too. Thanks a lot! Can we somehow add a note or full example to https://helpcenter.onlyoffice.com/server/document/document-server-proxy.aspx to save people's time? The Traefik sample is still for version 1

[kungknut]

I can also confirm that adding the X-Forwarded-Proto header solved my issues.

[SuperSandro2000]

I can confirm that only the X-Forwarded-Proto Header is needed.

[0x9060]

Similarly if you're having CORS problems, use the CORS header as well

traefik.http.routers.onlyoffice-secure.middlewares: "onlyoffice-sslheaders, onlyoffice-cors"
traefik.http.middlewares.onlyoffice-sslheaders.headers.customrequestheaders.X-Forwarded-Proto: "https"
traefik.http.middlewares.onlyoffice-cors.headers.accessControlAllowOrigin: "*"

That's working for me, too. Thank you so much!

My docker-compose config:

version: '3'

services:
  onlyoffice:
    container_name: onlyoffice
    image: onlyoffice/documentserver:latest
    restart: unless-stopped
    stdin_open: true
    tty: true
    volumes:
      - ${LOCAL_CONF_DIR}/onlyoffice:/var/log/onlyoffice
    environment:
      JWT_ENABLED: "true"
      JWT_SECRET: ${SECRET}
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.${HOSTNAME}-http.entrypoints=web"
      - "traefik.http.routers.${HOSTNAME}-http.rule=Host(`${HOSTNAME}.${DOMAIN0}`)"
      - "traefik.http.routers.${HOSTNAME}-http.middlewares=https_redirect@file"
      - "traefik.http.routers.${HOSTNAME}-https.entrypoints=websecure"
      - "traefik.http.routers.${HOSTNAME}-https.rule=Host(`${HOSTNAME}.${DOMAIN0}`)"
      - "traefik.http.routers.${HOSTNAME}-https.tls=true"
      - "traefik.http.routers.${HOSTNAME}-https.middlewares=sts@file,onlyoffice-headers"
      - "traefik.http.services.${HOSTNAME}.loadbalancer.server.port=80"

      ## Middleware definition
      # Headers for onlyoffice, https://github.com/ONLYOFFICE/onlyoffice-nextcloud/issues/151
      - "traefik.http.middlewares.onlyoffice-headers.headers.customrequestheaders.X-Forwarded-Proto=https"
      - "traefik.http.middlewares.onlyoffice-headers.headers.accessControlAllowOrigin=*"

networks:
  default:
    external:
      name: ${NETWORK}

This config is not working for me... I'm still getting (Connection refused) while connecting to upstream,

[jcgruenhage]

So, I've digged a bit, and the (merged) PR to traefik that @SuperSandro2000 linked above does not fix it. I refactored the code in traefik again to make it a bit clearer what's actually happening, but it doesn't make a difference really.

This needs to be fixed in ONLYOFFICE, but this is not the appropriate repository for that issue. Will open one though.

[zilexa]

I use FileRun (free, max 10 users, not opensource) instead of NextCloud, but I had the same issue with OnlyOffice.

For future reference, none of the workaround examples above are actually complete or are correct. After a few hours of trial and error, this is how I got it working. I spend DAYS figuring this out as I am a Traefik and OO noob. And this Traefik forum topic send me searching in the dark wasting even more time: https://community.traefik.io/t/traefik-2-0-and-onlyoffice-not-work-work-correctly-in-v-2/3286

After I had https://office.mydomain working, I tested the example doc file. When that worked, I started removing several Traefik labels until I got the bare minimum set of extra rules necessary to run OnlyOffice successfully with Traefikv2.0.

##_____________________ OnlyOffice Document Server [Cloud/Office]
  onlyoffice:
    image: onlyoffice/documentserver
    container_name: onlyoffice
    depends_on:
      - onlyoffice-rabbitmq
    stdin_open: true
    restart: always
    tty: true
    ports:
      - "8889:80"
    volumes:
      - $USERDIR/docker/onlyoffice/data:/var/www/onlyoffice/Data
      - $USERDIR/docker/onlyoffice/log:/var/log/onlyoffice
      - $USERDIR/docker/onlyoffice/cache:/var/lib/onlyoffice/documentserver/App_Data/cache/files
      - $USERDIR/docker/onlyoffice/example:/var/www/onlyoffice/documentserver-example/public/files
      - $USERDIR/docker/onlyoffice/fonts:/usr/share/fonts
    dns: 1.1.1.1
    environment:
      - JWT_ENABLED="true"
      - JWT_SECRET=$ONLYOFFICEJWT
      - AMQP_URI=amqp://guest:guest@onlyoffice-rabbitmq
    labels:
     ## My standard traefikv2.0 labels for services exposed online:
      - traefik.enable=true
      - traefik.http.routers.office-redirect.entrypoints=web
      - traefik.http.routers.office-redirect.rule=Host(`office.$DOMAIN`)
      - traefik.http.middlewares.office-redirect.redirectscheme.scheme=https
      - traefik.http.routers.office.middlewares=office-redirect
      - traefik.http.routers.office-secure.entrypoints=websecure
      - traefik.http.routers.office-secure.rule=Host(`office.$DOMAIN`)
      - traefik.http.routers.office.tls.certresolver=letsencrypt
      - traefik.http.services.office.loadbalancer.server.port=80
      ## Extra labels for onlyoffice:
      - traefik.http.routers.office-secure.tls=true
      - traefik.http.routers.office-secure.middlewares=secure-headers
      - traefik.http.middlewares.secure-headers.headers.customrequestheaders.X-Forwarded-Proto=https
      - traefik.http.middlewares.cors-headers.headers.accessControlAllowOrigin=*
      - traefik.http.routers.office.middlewares=secure-headers,cors-headers

      ## tested extra labels, works fine without DO NOT USE
      #- traefik.http.middlewares.office-redirectregex.redirectregex.regex=^http://(.*)
      #- traefik.http.middlewares.office-redirectregex.redirectregex.replacement=https://$$1
      #- traefik.http.middlewares.secure-headers.headers.referrerPolicy=no-referrer
      #- traefik.http.middlewares.secure-headers.headers.stsSeconds=31536000
      #- traefik.http.middlewares.secure-headers.headers.forceSTSHeader=true
      #- traefik.http.middlewares.secure-headers.headers.stsPreload=true
      #- traefik.http.middlewares.secure-headers.headers.stsIncludeSubdomains=true
      #- traefik.http.middlewares.secure-headers.headers.browserXssFilter=true

Also: I accidentally removed my postgreSQL container, discovered OnlyOffice still worked without it !? Not sure how, but now OnlyOffice runs, just with rabbitmq as additional container. Perhaps the onlyoffice/documentserver image already contains a sql database:

##____________________ Onlyoffice rabbitmq [CLOUD/Office]
  onlyoffice-rabbitmq:
    container_name: onlyoffice-rabbitmq
    image: rabbitmq
    restart: always
    expose:
      - '5672'

[chmanie]

@wcdgit and @calvinbui thanks so much for sharing your configs! Sadly these are still not working for me. This is my current config:

  onlyoffice-document-server:
    container_name: nextcloud-onlyoffice
    image: onlyoffice/documentserver:latest
    restart: always
    expose:
      - '80'
      - '443'
    volumes:
      - document_data:/var/www/onlyoffice/Data
      - document_log:/var/log/onlyoffice
    networks:
      - traefik_default
    labels:
      - traefik.enable=true
      - traefik.http.routers.onlyoffice-document-server.rule=Host(`-SNIP-`)
      - traefik.http.routers.onlyoffice-document-server.entrypoints=web
      - traefik.http.routers.onlyoffice-document-server.middlewares=https-redirect
      - traefik.http.routers.onlyoffice-document-server-https.rule=Host(`-SNIP-`)
      - traefik.http.routers.onlyoffice-document-server-https.entrypoints=websecure
      - traefik.http.routers.onlyoffice-document-server-https.tls=true
      - traefik.http.routers.onlyoffice-document-server-https.tls.certresolver=letsencrypt
      - traefik.http.routers.onlyoffice-document-server-https.middlewares=onlyoffice-headers
      - traefik.http.middlewares.onlyoffice-headers.headers.customrequestheaders.X-Forwarded-Proto=https
      - traefik.http.middlewares.onlyoffice-headers.headers.accessControlAllowOrigin=*

This is the config of the nextcloud app:

Here's the config for the nginx between traefik and nextcloud (and onlyoffice): https://gist.github.com/chmanie/3411b3533bbcfd3dd55a33a18accd31f

Am I missing anything?

EDIT: Commenting out the link as mentioned here worked for me, but I don't like this as a permanent solution.

[zilexa]

Just fyi since I posted my Traefik solution I switched to Caddyv2 as it is extremely simplified compared to Traefik. This works fine:

https://github.com/zilexa/Homeserver/blob/master/docker/docker-compose.yml#L279 With the caddy-docker-proxy container (also in that compose example). No other configuration is needed.

I also have a fully tested example with Nextcloud instead of FileRun: https://github.com/zilexa/Homeserver/blob/master/docker/Extras/nextcloud.yml

Both cases A+ security rating instantly.

[MTRNord]

Any update on this? Neither the custom request header nor removing the line in the nginx config works for me using the onlyoffice helm setup and a traefik2 ingress. :/