Degraded functionality when using `HIDE_REFERER` option

[mpeter50]

Describe the bug If I set the HIDE_REFERER environment variable to true, not only password login, but also core features like adding a new watch or setting the filter will not work anymore, as I will get the error described in #1255.

By setting this to false, the web interface will work as expected. However that is not a solution, because as I understand, this way the service will let external services to know about the hostaname of the changedetection service ran by the user. On one hand this is leaking information that may be sensitive, but the monitored website may also restrict access if they determine that these are automated requests done by a change detection and logging software.

Version v0.41

To Reproduce

Steps to reproduce the behavior:

1. run changedetection.io with the HIDE_REFERER=true environment variable

! ALWAYS INCLUDE AN EXAMPLE URL WHERE IT IS POSSIBLE TO RE-CREATE THE ISSUE - USE THE 'SHARE WATCH' FEATURE AND PASTE IN THE SHARE-LINK!

Expected behavior Password protection, adding watches, setting watch filters and other features should work without leaking information to the monitored websites.

Screenshots

Desktop (please complete the following information):

• OS: Windows
• Browser Firefox
• Version v0.41

Additional context It seems as if with HIDE_REFERER=true, GET requests to the changedetection service are always served as expected, and all POST requests consistently return with HTTP 400.

[masterwishx]

i have same problem but after login to main page (after password entered) - > 400 Bad Request behind nginx reverse proxy and this is after enabled HIDE_REFERER , if disabled then all Fine ....

https://github.com/dgtlmoon/changedetection.io/discussions/1322#discussion-4779729 https://github.com/dgtlmoon/changedetection.io/discussions/1255#discussioncomment-5671706

[benmargolin]

THANK YOU! I was seeing this problem manifest as the playwright stuff timing out, turns out it was the referer issue. I changed it to - HIDE_REFERER=false (I run with SSL behind Caddy) and all works again. But I agree this is theoretically not the best solution, either.

[TheNeikos]

I also have this problem, and can't change anything unless I disable HIDE_DEFERER. I am also behind an nginx proxy

[87435]

Same here. Selfhosted on https://cd.domain.tld, served using Caddy using reverse_proxy :5000 leads to the pages being seemingly working (browsable) however trying to add a new watch, save settings or pretty much sending any form will give me 400 Error - Bad Request - The referrer header is missing. unless I disable HIDE_REFERER, which is not the behaviour I want.

[dgtlmoon]

I think actually that was a bad PR, and the author of the PR has gone pretty much missing.

unfortunately this is why many opensource projects shouldnt accept PR's from people :-(

I think the correct solution is to remove HIDE_REFERER, and then use some kind of "rel=noreferer" on any external links

https://developer.mozilla.org/en-US/docs/Web/HTML/Attributes/rel/noreferrer

thoughts?

[87435]

That looks like a good option, although this is not really my field and I don't know enough to understand if there are other implications to doing this.

TBH I am surprised the HIDE_REFERER has an impact at all on the changedetection app itself, shouldn't this setting apply only when making "external" requests? If I understand correctly this is the only relevant part, does that mean that it is the same Flask app that provides the web interface and fetches external data?

[87435]

If we take things from another angle (and again this might reveal how ignorant I am on the topic): is there a reason why the "internal" app (ie the web interface for watches, settings etc) is so strict about referer?

[dgtlmoon]

TBH I am surprised the HIDE_REFERER has an impact at all on the changedetection app itself, shouldn't this setting apply only when making "external" requests?

No, its because the referer is used to calculate the XSS form protection checksum and so it gets broken because its missing, therefor all form stuff breaks/invalidates.

it is not compatible

If we take things from another angle (and again this might reveal how ignorant I am on the topic): is there a reason why the "internal" app (ie the web interface for watches, settings etc) is so strict about referer?

@87435 see above

[enviousjag]

Are there any updates on this issue? I gather the current workaround is to disable HIDE_REFFER?