Recognize wget as a terminal application

Myzel394 commented 2 weeks ago

Recognize wget like curl

Example user agent:

Wget/1.24.5

gomarcd commented 2 weeks ago

Hi @Myzel394, thank you for the feedback.

What is the intended use case/outcome - did you want wget to save the plaintext response similar to what curl returns?

I just did a quick update, take a look and let me know if that is what you were after.

$ wget ip.now
Prepended http:// to 'ip.now'
--2024-11-19 07:04:41--  http://ip.now/
Resolving ip.now (ip.now)... 104.21.16.228, 172.67.216.130
Connecting to ip.now (ip.now)|104.21.16.228|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://ip.now/ [following]
--2024-11-19 07:04:41--  https://ip.now/
Connecting to ip.now (ip.now)|104.21.16.228|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 14 [text/plain]
Saving to: ‘index.html’

index.html                100%[===================================>]      14  --.-KB/s    in 0s      

2024-11-19 07:04:41 (621 KB/s) - ‘index.html’ saved [14/14]

$ cat index.html
106.168.44.86

And:

$ wget ip.now/details

Prepended http:// to 'ip.now/details'
--2024-11-19 07:05:42--  http://ip.now/details
Resolving ip.now (ip.now)... 104.21.16.228, 172.67.216.130
Connecting to ip.now (ip.now)|104.21.16.228|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://ip.now/details [following]
--2024-11-19 07:05:42--  https://ip.now/details
Connecting to ip.now (ip.now)|104.21.16.228|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 289 [application/json]
Saving to: ‘details’

details                   100%[===================================>]     289  --.-KB/s    in 0s      

2024-11-19 07:05:42 (23.0 MB/s) - ‘details’ saved [289/289]

$ cat details
{
  "ip": "106.168.44.86",
  "isp": "ISP Inc.",
  "asn": 123,
  "location": "Realtown, SomeState, Country, Code",
  "latitude": "113.53120",
  "longitude": "-122.12340",
  "datacenter": "YUL",
  "userAgent": "Wget/1.25.0",
  "timezone": "Time/Zone",
  "tlscipher": "AEAD-AES256-GCM-SHA384"
}

Myzel394 commented 2 weeks ago

Oh right, I just tried it and can confirm this works. Weird, the other day it would return the website instead of just the ip, but you probably fixed it in the meantime. :D

BTW, what are your log policies for your server?

gomarcd commented 1 week ago

By default it would return the whole html, I think it's just how wget works. I updated it to return only the same plain text response if wget is in the user agent. Glad it's looking better for you now.

Good question on the logs, I made this thing out of curiosity for learning purposes so it's deployed as a Cloudflare Worker on a free tier for the time being rather than on any dedicated server. I can see in their dashboard it gives me up to 3 days for Worker logs ("Upgrade to paid plan to see more!"), and up to 30 days of general web traffic metrics for the domain.

You can read more about what is available to Cloudflare out of the box on every request here: https://developers.cloudflare.com/workers/runtime-apis/request/#incomingrequestcfproperties.

Myzel394 commented 1 week ago

By default it would return the whole html, I think it's just how wget works. I updated it to return only the same plain text response if wget is in the user agent. Glad it's looking better for you now.

Perfect! Then this issue is resolved

deployed as a Cloudflare Worker on a free tier for the time being rather than on any dedicated server

Ah too bad, unfortunately this is not suitable for us then :/ We need to rely on services that do not log

gomarcd / ipnow

Recognize wget as a terminal application #2