Python Actors: Use `follow_redirects=True` when HTTPX is used

apify / actor-templates

This project is the :house: home of Apify actor template projects to help users quickly get started.

https://apify.com/

24 stars 14 forks source link

Closed vdusek closed 9 months ago

vdusek commented 9 months ago

I discovered that HTTPX defaults to follow_redirects=False.

For instance, if a user specifies https://www.apify.com/ as a starting URL, the resulting output will be as follows:

{
  "url": "https://www.apify.com/",
  "title": "301 Moved Permanently"
}

This behavior might not be optimal. In most cases, you would want to follow 301 (or any 3xx) redirections.

vdusek commented 9 months ago