502 Bad Gateway

[chf0x]

Hi guys. Thank you for the great project. After setting everything up and following the readme, I am getting a 502 Bad Gateway error when trying to reach the login page. I can confirm that MongoDB and Redis are started and working properly. In the Docker Compose logs, there are no errors on start, and I can see that the database is replicated properly.

I found the following error in the Nginx error logs:

2024/04/20 22:33:57 [error] 178#178: *1 connect() failed (111: Connection refused) while connecting to upstream, client: 172.18.0.9, server: _, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:4000/", host: "overleaf.domain.com"

Then I went into the Docker container and found that indeed there is nothing on port 4000:

root@21f28f798e58:/overleaf/services/web# netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 127.0.0.1:3003          0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:3036          0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:3005          0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:3009          0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:3010          0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:3042          0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:3013          0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:3016          0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:3048          0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:3049          0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:3054          0.0.0.0:*               LISTEN      -                   
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      163/nginx: master p 
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN      163/nginx: master p 
tcp        0      0 127.0.0.11:34353        0.0.0.0:*               LISTEN      -                   
tcp        0      0 127.0.0.1:3026          0.0.0.0:*               LISTEN      -                   
tcp6       0      0 :::10843                :::*                    LISTEN      -                   
tcp6       0      0 :::3100                 :::*                    LISTEN      -                   
udp        0      0 127.0.0.11:51903        0.0.0.0:*                           -                  

Here is what I found in /var/log/sharelatex/web.log. Could this error be related to my issue?

Node.js v18.18.2
Initializing metrics
Set UV_THREADPOOL_SIZE=16
Using default settings from /overleaf/services/web/config/settings.defaults.js
Using settings from /etc/sharelatex/settings.js
/overleaf/node_modules/express/lib/router/route.js:211
        throw new Error(msg);
        ^

Error: Route.post() requires a callback function but got an [object Undefined]
    at Route.<computed> [as post] (/overleaf/node_modules/express/lib/router/route.js:211:15)
    at proto.<computed> [as post] (/overleaf/node_modules/express/lib/router/index.js:521:19)
    at Object.initialize (/overleaf/services/web/app/src/router.js:745:13)
    at Object.<anonymous> (/overleaf/services/web/app/src/infrastructure/Server.js:340:8)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Module.require (node:internal/modules/cjs/loader:1143:19)
    at require (node:internal/modules/cjs/helpers:119:18)
    at Object.<anonymous> (/overleaf/services/web/app.js:31:16)
    at Module._compile (node:internal/modules/cjs/loader:1256:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1310:10)
    at Module.load (node:internal/modules/cjs/loader:1119:32)
    at Module._load (node:internal/modules/cjs/loader:960:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:86:12)

Thank you in advance! And please let me know if I can provide any additional logs?

[smhaller]

hm, what are your settings in the .env file? can you post your configuration the docker-compose file thx

[chf0x]

version: '3.7'
networks:
  overleaf-external:
    name: webproxy
    external: true
  overleaf-local:
    driver: bridge

services:
  web:
    restart: always
    build: .
    container_name: overleaf_sharelatex
    depends_on:
      mongo:
        condition: service_started
      redis:
        condition: service_started
    links:
      - mongo
      - redis
    networks:
      - "overleaf-local"
      - "overleaf-external"
    volumes:
      - ./nginx_logs:/var/log/nginx/
      - ./sharelatex_data:/var/lib/sharelatex
      - ./sharelatex_packages:/usr/local/texlive
    environment:
      SHARELATEX_APP_NAME: [CUSTOM APP NAME]
      SHARELATEX_MONGO_URL: mongodb://[MONGO_URL]/sharelatex
      SHARELATEX_REDIS_HOST: redis
      REDIS_HOST: redis
      ENABLE_CONVERSIONS: 'true'
      EMAIL_CONFIRMATION_DISABLED: 'true'
      SHARELATEX_SITE_URL: [CUSTOM SITE URL]
      SHARELATEX_NAV_TITLE: [CUSTOM NAV TITLE]
      SHARELATEX_ADMIN_EMAIL: [ADMIN EMAIL]
      SHARELATEX_EMAIL_FROM_ADDRESS: "[FROM EMAIL ADDRESS]"
      SHARELATEX_EMAIL_SMTP_HOST: [SMTP HOST]
      SHARELATEX_EMAIL_SMTP_PORT: [SMTP PORT]
      SHARELATEX_EMAIL_SMTP_SECURE: 'true'
      SHARELATEX_EMAIL_SMTP_USER: [SMTP USER]
      SHARELATEX_EMAIL_SMTP_PASS: [SMTP PASSWORD]
      SHARELATEX_ALLOW_ANONYMOUS_READ_AND_WRITE_SHARING: 'true'
      SHARELATEX_ALLOW_PUBLIC_ACCESS: 'true'
      SHARELATEX_SECURE_COOKIE: "true"
      SHARELATEX_BEHIND_PROXY: "true"
      OAUTH2_ENABLED: "true"
      OAUTH2_AUTHORIZATION_CONTENT_TYPE: "application/x-www-form-urlencoded"
      OAUTH2_CLIENT_ID: [CLIENT ID]
      OAUTH2_CLIENT_SECRET: [CLIENT SECRET]
      OAUTH2_AUTHORIZATION_URL: [AUTHORIZATION URL]
      OAUTH2_TOKEN_URL: [TOKEN URL]
      OAUTH2_PROFILE_URL: [PROFILE URL]
      OAUTH2_SCOPE: ""
      OAUTH2_USER_ATTR_EMAIL: "email"
      OAUTH2_USER_ATTR_UID: "email"
      OAUTH2_USER_ATTR_FIRSTNAME: "name"
      OAUTH2_USER_ATTR_IS_ADMIN: "is_admin"
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.overleaf-secured.rule=Host(`[CUSTOM HOST]`)"
      - "traefik.http.routers.overleaf-secured.tls=true"
      - "traefik.http.routers.overleaf-secured.tls.certresolver=myresolver"
      - "traefik.http.routers.overleaf-secured.entrypoints=websecure"
      - "traefik.http.services.overleaf-secured.loadbalancer.server.port=80"
      - "traefik.http.routers.overleaf-secured.middlewares=chain-authentik@file,overleaf-sso"
      - "traefik.http.middlewares.overleaf-sso.replacepathregex.regex=^/login"
      - "traefik.http.middlewares.overleaf-sso.replacepathregex.replacement=/oauth/redirect"
      - "traefik.docker.network=webproxy"

  mongo:
    restart: always
    image: mongo:4.4
    container_name: overleaf_mongo
    command: "--replSet overleaf"
    volumes:
      - "./mongo_data:/data/db"
    healthcheck:
      test: "mongo --quiet --eval 'rs.hello().setName ? rs.hello().setName : rs.initiate({_id: \"overleaf\",members:[{_id: 0, host:\"mongo:27017\"}]})'"
      interval: 10s
      timeout: 10s
      retries: 5
    networks:
      - "overleaf-local"

  redis:
    restart: always
    image: redis:latest
    container_name: overleaf_redis
    networks: 
      - "overleaf-local"
    volumes:
      - ./redis_data:/data

Here is my Docker Compose file. Everything was functioning until I upgraded Docker. I attempted to revert to the previous version, but it no longer works.

docker -v
Docker version 26.1.2, build 211e74b

[chf0x]

I tried to update /etc/nginx/sited-enabled/sharelatex.conf with this https://github.com/overleaf/overleaf/pull/1175/commits/25145708b8fa5f355ed069ead4fafcb0d8f9cdd0 But still 502 error unforunately. Also I tried to update to 4.2.4 as suggested in https://github.com/overleaf/overleaf/issues/1176#issuecomment-2095560792. With the same result. So I guess, something is wrong with my docker-compose...

[chf0x]

It seems that overleaf is just not starting. When I try to run it manually in the container I get:

root@6c12032214e1:/overleaf/services/web# npm start

> start
> node $NODE_APP_OPTIONS app.js

Initializing metrics
Set UV_THREADPOOL_SIZE=16
Using default settings from /overleaf/services/web/config/settings.defaults.js
Using settings from /etc/sharelatex/settings.js
express-session deprecated req.secret; provide secret option app/src/infrastructure/Server.js:151:3
{"name":"web","hostname":"6c12032214e1","pid":11021,"level":50,"filename":"/overleaf/services/web/app/views/user/settings.pug","err":{"stack":"(no stack)","info":{}},"msg":"error compiling","time":"2024-05-13T12:03:35.227Z","v":0}
/overleaf/node_modules/express/lib/router/route.js:216
        throw new Error(msg);
        ^

Error: Route.post() requires a callback function but got a [object Undefined]
    at Route.<computed> [as post] (/overleaf/node_modules/express/lib/router/route.js:216:15)
    at proto.<computed> [as post] (/overleaf/node_modules/express/lib/router/index.js:521:19)
    at Object.initialize (/overleaf/services/web/app/src/router.js:745:13)
    at Object.<anonymous> (/overleaf/services/web/app/src/infrastructure/Server.js:340:8)
    at Module._compile (node:internal/modules/cjs/loader:1364:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Module._load (node:internal/modules/cjs/loader:1019:12)
    at Module.require (node:internal/modules/cjs/loader:1231:19)
    at require (node:internal/modules/helpers:177:18)
    at Object.<anonymous> (/overleaf/services/web/app.js:31:16)
    at Module._compile (node:internal/modules/cjs/loader:1364:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1422:10)
    at Module.load (node:internal/modules/cjs/loader:1203:32)
    at Module._load (node:internal/modules/cjs/loader:1019:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:128:12)

Node.js v18.20.2
npm error Lifecycle script `start` failed with error:
npm error Error: command failed
npm error   in workspace: @overleaf/web
npm error   at location: /overleaf/services/web

[chf0x]

I can confirm that vanilla sharelatex 4.2 or 4.2.4 container is working properly

[smhaller]

Can you post the Dockerfile you are using as well? If you use the repo version did you execute the following?

bash scripts/extract_files.sh 4.2.0
bash scripts/apply_diffs.sh
make

[chf0x]

Dockerfile:

FROM sharelatex/sharelatex:4.2
#FROM sharelatex/sharelatex:latest
# latest might not be tested 
# e.g. the AuthenticationManager.js script had to be adapted after versions 2.3.1 
LABEL maintainer="Simon Haller-Seeber"
LABEL version="0.1"

# passed from .env (via make)
ARG collab_text
ARG login_text   
ARG admin_is_sysadmin

# set workdir (might solve issue #2 - see https://stackoverflow.com/questions/57534295/)
WORKDIR /overleaf/services/web

    # install latest npm
RUN npm install -g npm && \
    ## clean cache (might solve issue #2)
    # npm cache clean --force && \
    npm install ldap-escape ldapts-search ldapts@3.2.4 && \
    # npm install bcrypt@5.0.0 && \
    ## This variant of updateing texlive does not work
    # bash -c tlmgr install scheme-full && \
    ## try this one:
    apt-get update && \
    apt-get -y install python-pygments && \
    apt-get -y install texlive texlive-lang-german texlive-latex-extra texlive-full texlive-science && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/*

# overwrite some files
COPY sharelatex/AuthenticationManager.js    /overleaf/services/web/app/src/Features/Authentication/
COPY sharelatex/AuthenticationController.js /overleaf/services/web/app/src/Features/Authentication/
COPY sharelatex/ContactController.js        /overleaf/services/web/app/src/Features/Contacts/
COPY sharelatex/router.js                   /overleaf/services/web/app/src/router.js

# Too much changes to do inline (>10 Lines).
COPY sharelatex/settings.pug    /overleaf/services/web/app/views/user/
COPY sharelatex/login.pug       /overleaf/services/web/app/views/user/
COPY sharelatex/navbar.pug      /overleaf/services/web/app/views/layout/

# Non LDAP User Registration for Admins
COPY sharelatex/admin-index.pug     /overleaf/services/web/app/views/admin/index.pug
COPY sharelatex/admin-sysadmin.pug  /tmp/admin-sysadmin.pug

    ## comment out this line to prevent sed accidently remove the brackets of the email(username) field
    # sed -iE '/email@example.com/{n;N;N;d}' /overleaf/services/web/app/views/user/login.pug && \
RUN sed -iE "s/email@example.com/${login_text:-user}/g" /overleaf/services/web/app/views/user/login.pug && \
    ## Collaboration settings display (share project placeholder) | edit line 146
    ## share.pug file was removed in later versions
    # sed -iE "s%placeholder=.*$%placeholder=\"${collab_text}\"%g" /overleaf/services/web/app/views/project/editor/share.pug && \
    ## extend pdflatex with option shell-esacpe ( fix for closed overleaf/overleaf/issues/217 and overleaf/docker-image/issues/45 )
    ## do this in different ways for different sharelatex versions
    sed -iE "s%-synctex=1\",%-synctex=1\", \"-shell-escape\",%g" /overleaf/services/clsi/app/js/LatexRunner.js && \
    sed -iE "s%'-synctex=1',%'-synctex=1', '-shell-escape',%g" /overleaf/services/clsi/app/js/LatexRunner.js && \
    if [ "${admin_is_sysadmin}" = "true" ] ; \
        then cp /tmp/admin-sysadmin.pug /overleaf/services/web/app/views/admin/index.pug ; \
        else rm /tmp/admin-sysadmin.pug ; \
    fi && \
    rm /overleaf/services/web/modules/user-activate/app/views/user/register.pug && \
    ### To remove comments entirly (bug https://github.com/overleaf/overleaf/issues/678)
    rm /overleaf/services/web/app/views/project/editor/review-panel.pug && \
    touch /overleaf/services/web/app/views/project/editor/review-panel.pug

### Nginx and Certificates
# enable https via letsencrypt
# RUN rm /etc/nginx/sites-enabled/sharelatex.conf
# COPY nginx/sharelatex.conf /etc/nginx/sites-enabled/sharelatex.conf

# get maintained best practice ssl from certbot
# RUN wget https://raw.githubusercontent.com/certbot/certbot/master/certbot-nginx/certbot_nginx/_internal/tls_configs/options-ssl-nginx.conf -O /etc/nginx/options-ssl-nginx.conf && \
#     wget https://raw.githubusercontent.com/certbot/certbot/master/certbot/certbot/ssl-dhparams.pem -O /etc/nginx/ssl-dhparams.pem 

# reload nginx via cron for reneweing https certificates automatically
# COPY nginx/nginx-reload.sh  /etc/cron.weekly/
# RUN chmod 0744 /etc/cron.weekly/nginx-reload.sh

## extract certificates from acme.json?
# COPY nginx/nginx-cert.sh /etc/cron.weekly/
# RUN chmod 0744 /etc/cron.weekly/nginx-cert.sh && \
#     echo "/usr/cron.weekly/nginx-cert.sh 2>&1 > /dev/null" > /etc/rc.local && \
#     chmod 0744 /etc/rc.local

And yes, I did run that scripts. Just in case what is you docker version?

[smhaller]

I have instances running with: docker version: 24.0.7 and 25.0.3 and for docker compose versions: 1.29.2, 2.21.0 and 2.24.5

hm, did you extract the files for exactly the same version you use in the Dockerfile?

[maurerle]

I somehow have problems starting my running overleaf container since I updated docker to 26.1.4.. Maybe this is related, I did not find the issue yet. In /var/log/sharelatex/web.log i did only find the "Please migrate your code to use AWS SDK for JavaScript (v3)." stuff.

[maurerle]

For me the only thing which helped was to downgrade Docker back to the previous version:

sudo apt install docker-ce-cli=5:25.0.3-1~debian.12~bookworm containerd.io:amd64=1.6.28-1 docker-ce=5:25.0.3-1~debian.12~bookworm.

I did get the normal sharelatex:5 container running on the latest docker version though - so we should find a upgrade way to sharelatex:5 using ldap soon :)

[maurerle]

The reason for this is ipv6 support in docker 26. See here for a fix without downgrading (not tested myself): https://github.com/overleaf/overleaf/issues/1146#issuecomment-1709237699

[davidliyutong]

The reason for this is ipv6 support in docker 26. See here for a fix without downgrading (not tested myself): overleaf/overleaf#1146 (comment)

For me, I have encountered the same 502 Bad Gateway error and fixed it following this guide - my instance is deployed on K8S