dgtlmoon / changedetection.io

The best and simplest free open source web page change detection, website watcher, restock monitor and notification service. Restock Monitor, change detection. Designed for simplicity - Simply monitor which websites had a text change for free. Free Open source web page change detection, Website defacement monitoring, Price change notification
https://changedetection.io
Apache License 2.0
15.84k stars 884 forks source link

Basic fast Plaintext/HTTP Client: `User-Agent/user-agent` not working #2384

Closed leiless closed 1 month ago

leiless commented 1 month ago

Describe the bug

Set User-Agent/user-agent request header in Basic fast Plaintext/HTTP Client is not working.

Version v0.45.23

Expected behaviour The Basic fast Plaintext/HTTP Client should honor the User-Agent / user-agent, but itsn't.

To Reproduce

Steps to reproduce the behavior:

  1. Paste https://httpbin.org/headers and click Edit/Watch
  2. Set to use the Basic fast Plaintext/HTTP Client
  3. Click Show advanced options
  4. Set Request headers:
    User-Agent: curl/7.88.0
  5. Set CSS/JSONPath/JQ/XPath Filters to jq: .headers |= del(.["X-Amzn-Trace-Id"])
  6. Click Save
{
    "headers": {
        "Accept": "*/*",
        "Accept-Encoding": "gzip, deflate, br",
        "Host": "httpbin.org",
        "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.66 Safari/537.36"
    }
}

Additional context

I've also tried user-agent: curl/7.88.0, still not working.

user-agent: curl/7.88.0
User-Agent: curl/7.77.0

Also not working.


User-Agent: curl/7.77.0
user-agent: curl/7.88.0

It's weird but working, the output user-agent is the curl/7.88.0.

image

dgtlmoon commented 1 month ago

thanks!

after re-reading and re-reading your post, i guess you're saying it works but only when lowercase user-agent or? i dont quite follow

image

works OK..

image

leiless commented 1 month ago

I'm using a self-hosted (Linux aarch64) instance. And after I restarted my instance, this problem still exists.

Not working case 1

User-Agent: curl/7.77.0

Not working case 2

user-agent: curl/7.88.0

Not working case 3

user-agent: curl/7.88.0
User-Agent: curl/7.77.0

(lower-cased first)

Working but weird case 1

User-Agent: curl/7.77.0
user-agent: curl/7.88.0

(upper-cased first)

The output user-agent in change detection is curl/7.88.0.

dgtlmoon commented 1 month ago

@leiless can you click the 'share watch' icon and share the watch settings? i am totally unable to reproduce this

dgtlmoon commented 1 month ago

cannot reproduce

dgtlmoon commented 3 weeks ago

I see now :)

leiless commented 3 weeks ago

So what is the root cause by now?

dgtlmoon commented 3 weeks ago

@leiless look at the PR https://github.com/dgtlmoon/changedetection.io/pull/2409/files

leiless commented 2 weeks ago

Fixed by https://github.com/dgtlmoon/changedetection.io/releases/tag/0.45.24