dgtlmoon / changedetection.io

The best and simplest free open source web page change detection, website watcher, restock monitor and notification service. Restock Monitor, change detection. Designed for simplicity - Simply monitor which websites had a text change for free. Free Open source web page change detection, Website defacement monitoring, Price change notification
https://changedetection.io
Apache License 2.0
17.3k stars 965 forks source link

Reverse proxy - Password protection randomly disables when clicking logout #598

Closed kzshantonu closed 2 years ago

kzshantonu commented 2 years ago

Describe the bug Not exactly sure what causes this. It's very random and does not always happen. Cannot find anything about this in the logs.

Version v0.39.13.1

ginebib commented 2 years ago

Same problem here, but happens everytime: When I set a password, I immediately have to enter it after that. But next time I visit the page, there's no password protection.

Everything else works fine.

Version v0.39.13.1 Installed with docker-compose

dgtlmoon commented 2 years ago

@kzshantonu please describe your steps 1.. 2.. 3...

ginebib commented 2 years ago

I tried it now with version v0.39.13. Password protection works as expected here. So it seems to be a bug only in v0.39.13.1. It's a fresh installation with docker-compose. As a reverse proxy I use Traefik. Here is my docker-compose.yml

services:
    changedetection:
      image: ghcr.io/dgtlmoon/changedetection.io # :0.39.13 works fine
      container_name: changedetection
      hostname: changedetection
      volumes:
        - changedetection-data:/datastore
      networks:
        - proxy
      restart: unless-stopped
      labels:
        - "traefik.enable=true"
        - "traefik.http.routers.changedetection.entrypoints=http"
        - "traefik.http.routers.changedetection.rule=Host(`changedetection.example.com`)"
        - "traefik.http.middlewares.changedetection-https-redirect.redirectscheme.scheme=https"
        - "traefik.http.routers.changedetection.middlewares=freshrss-https-redirect"
        - "traefik.http.routers.changedetection-secure.entrypoints=https"
        - "traefik.http.routers.changedetection-secure.rule=Host(`changedetection.example.com`)"
        - "traefik.http.routers.changedetection-secure.tls=true"
        - "traefik.http.routers.changedetection-secure.tls.certresolver=http"
        - "traefik.http.routers.changedetection-secure.service=changedetection"
        - "traefik.http.services.changedetection.loadbalancer.server.port=5000"
        - "traefik.docker.network=proxy"

volumes:
  changedetection-data:

networks:
    proxy:
      external: true

Steps to reproduce (at least with me):

  1. fresh install with docker-compose using the docker-compose.yml from above
  2. visit https://changedetection.example.com and set a password
  3. save
  4. i am redirected to password page --> enter password
  5. i am logged in <-- so far everything works as expected
  6. logout (or visit page later)
  7. i see immediately the main page; no password protection is set
dgtlmoon commented 2 years ago

@ginebib only difference in v0.39.13.1 is just a fix for the pip packager, if you didnt install via pip, then it makes no difference to you :-1:

trafeik - maybe the bug is that theres a header not acting correctly through your reverse proxy setup, keep digging! let me know what you find out

try this - i bet it works fine if you access the container directly, NOT through any reverse proxy - let me know what you find out

ginebib commented 2 years ago

Sounds strange, but I can't reproduce it now. I tried yesterday different configurations but everytime with the same behavior. Now I am back on v0.39.13.1, I changed nothing else, and it works now. When it happens again, I will have a deeper look at the traefik configuration.

For me this can be closed :) @dgtlmoon Thank you, great work!

dgtlmoon commented 2 years ago

@ginebib ok thanks, yeah it could still be an actual issue tho!

kzshantonu commented 2 years ago

I can confirm it's similar to what @ginebib described. I remember changing a setting. Then hit logout. Then it went straight to main page. Settings said I don't have a password set anymore. I am using caddy reverse proxy. Is there any extra header I need to pass?

dgtlmoon commented 2 years ago

not that i'm aware of - you guys will have to find the bugs on this i think, i dont have time to install caddy or traefik unfortunately

kzshantonu commented 2 years ago

So I finally reproduced the issue. Version: 0.39.13.1

  1. Set a password
  2. Hit save
  3. Login with the new password
  4. Go to settings > fetching
  5. Change fetch method
  6. Hit save
  7. Logout
  8. Goes to main screen
  9. Password gets disabled under settings
dgtlmoon commented 2 years ago

Unable to reproduce under nginx.. i dont use/know caddy or traefik sorry :(

xiya233 commented 2 years ago

Same problem, I use nginx, It's just that nginx doesn't run inside the container, it runs on the host

Here is my nginx.conf and docker-compose.yml

server {
    listen       80;
    server_name  monitor.example.com;
    client_max_body_size 0;

    location / {
        proxy_pass       http://127.0.0.1:6000/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
version: '3.5'

services:
  changedetection:
    image: ghcr.io/dgtlmoon/changedetection.io
    container_name: changedetection
    hostname: changedetection
    restart: unless-stopped
    environment:
      - USE_X_SETTINGS=1
      - BASE_URL=https://monitor.example.com/
      - PLAYWRIGHT_DRIVER_URL=ws://playwright-chrome:3000/
    ports:
      - 127.0.0.1:6000:5000
    volumes:
      - changedetection-data:/datastore

  playwright-chrome:
    image: browserless/chrome
    hostname: playwright-chrome
    restart: unless-stopped

volumes:
  changedetection-data:

update

I just tested it, and even without using a reverse proxy, I still have this problem

@dgtlmoon Maybe you should look into it

dgtlmoon commented 2 years ago

@dgtlmoon Maybe you should look into it

@xiya233 maybe you should pay me

sorry - this is free software - please provide your own time to help out here also, I'm working on this project in my spare time when I should be working a real job to better afford my own living costs

what gives you the right to demand that I "look into it", when the project is donated to you - for free?

please dig deeper

langfingaz commented 2 years ago

@dgtlmoon Thank you for taking the time to write this software.

Please excuse me, but I don't quite understand why you are so upset with @xiya233's post.

He gave us the information that he also has the problem with Nginx and that it happens even without a reverse proxy - so more users might be affected.

So he thought you might be interested in taking a closer look at the problem. I don't think he was demanding you to do that, though.

@xiya233 Can you reproduce the bug without a reverse proxy and describe your steps?

kzshantonu commented 2 years ago

@langfingaz the steps I outlined works with or without reverse proxy. For now I am setting the password again every time I change any setting

dgtlmoon commented 2 years ago

If someone would want to setup a complete docker-compose.yml with the setup that reproduces the problem, i'll take alook

but without this, i cant do anything, i need to see what the environment is and work with it

kzshantonu commented 2 years ago

services:
    changedetection:
      image: ghcr.io/dgtlmoon/changedetection.io
      container_name: changedetection
      hostname: changedetection
      volumes:
        - ./data:/datastore
      environment:
        - PORT=43571
        - PLAYWRIGHT_DRIVER_URL=ws://browserless:3000/
        - BASE_URL=https://changes.my.domain
      ports:
        - 100.109.179.81:43571:43571
      restart: unless-stopped
    playwright-chrome:
      hostname: browserless
      container_name: browserless
      image: browserless/chrome:1.53-chrome-stable
      restart: unless-stopped
      shm_size: 2g
      environment:
        - "DEFAULT_LAUNCH_ARGS=[\"--window-size=1920,1080\"]"
dgtlmoon commented 2 years ago

@kzshantonu there is no reverse proxy there

kzshantonu commented 2 years ago

I run caddy on my system as a systemd service. It doesn't matter. The issue happens even when accessed directly using IP:port

marconett commented 2 years ago

I can reproduce this too (on v0.39.17.1), and it has nothing to do with reverse proxies, logging out or changing settings.

The password protection disappears when clicking "save" in the settings.

Steps to reproduce:

It looks to me like a basic form handling issue.

When accessing the settings page, all form fields get pre-populated with the current settings from the store. If password protection is enabled, the password form field doesn't get pre-populated, because this would make the password readable for any logged in user: https://github.com/dgtlmoon/changedetection.io/blob/291700554ee83a2b39fcd0ac3d90efdea3d6cb83/changedetectionio/templates/settings.html#L46

This is good practice.

But when clicking save now, the form does not post a password field and changedetection.io interprets this as something like "if there was no password posted, remove password protection".

I don't know anything about wtforms and glancing at the code, I couldn't identify where the problem actually occurs, but I suspect it's near here: https://github.com/dgtlmoon/changedetection.io/blob/26f5c56ba40e3dda71f3d4cf2f5eb3b09331af34/changedetectionio/forms.py#L83

PotatoYummy commented 2 years ago

That's true. For me, the steps to reproduce the issue(v0.39.16) :

  1. Set a password
  2. Hit save
  3. Login with the new password
  4. Go to settings
  5. Hit save
  6. Open the site in a new tab
  7. Hit logout
  8. Password is gone
dgtlmoon commented 2 years ago

@PotatoYummy which reverse proxy? there are several, can you paste a config?

dgtlmoon commented 2 years ago

@PotatoYummy right! I can reproduce it without a reverse proxy!

Looks like hitting [save] in [settings] with a password set will sometimes do something strange with the password

kzshantonu commented 2 years ago

That's what I've been saying from the beginning. Glad you reproduced it

dgtlmoon commented 2 years ago

@kzshantonu but what you are saying has nothing todo with a reverse proxy

dgtlmoon commented 2 years ago

@kzshantonu @PotatoYummy @marconett please try the docker :dev tag when https://github.com/dgtlmoon/changedetection.io/actions/runs/2799910172 is finished

thanks for your patience, the mention of the reverse proxy really confused the situation - turned out it was completely unrelated

kzshantonu commented 2 years ago

Great. Works as expected now. Thanks for the wonderful project

dgtlmoon commented 2 years ago

Fixed in 0.39.17.2 which is out now