search

mirror / wget

Wget Git mirror
GNU General Public License v3.0
394 stars 134 forks source link

HTTP request sent, awaiting response... 404 Not Found on existing page #21

Open estatistics opened 2 years ago

estatistics commented 2 years ago

page exists, but not downloaded... i tried to put headers but not success. what to do? is that a bug? I am downloading wget2...

LANG=EN; wget --random-wait -4 --max-redirect 100 --level 100 --limit-rate=30K -r -nc -p -U Mozilla 'www.literotica.com/average-joe-ch-01?page=2'

URL transformed to HTTPS due to an HSTS policy --2022-07-11 13:49:09-- https://www.literotica.com/average-joe-ch-01?page=2 Resolving www.literotica.com (www.literotica.com)... 216.150.65.200, 216.150.65.190 Connecting to www.literotica.com (www.literotica.com)|216.150.65.200|:443... connected. HTTP request sent, awaiting response... 404 Not Found 2022-07-11 13:49:10 ERROR 404: Not Found.

estatistics commented 2 years ago

any link with "?page=xxxnumber" type could not downloaded by wget..... I think it can not follow redirections.