Infinite loop on self-refreshing pages

[kmowery]

Issue: Calling br.open(url) enters an infinite refresh loop if the page has a refresh header pointing to itself.

Reasons:

• The default arguments to HTTPRefreshProcessor follow all refreshes, after waiting the page-requested amount of time.
• Since the page is fetched correctly, adding a timeout parameter to open() does nothing. (Upon reflection, this seems like correct behavior to me. However, it is quite counter-intuitive to a library user, especially since there's no indication of why mechanize is hanging.)

In my case, I don't care about refresh headers, so I simply changed the default arguments at _useragent.py:107.

Possible Solutions:

• Allow customization of refresh header behavior in the Browser object.
• Ignore header refreshes after the browser timeout has passed.

Thoughts?

(Thanks for mechanize, btw, it's a fantastic piece of software!)

[jjlee]

• Are you aware of the about max_time and honor_time arguments to .set_handle_refresh()?
• I believe the maximum redirection count should apply here (10 total, 4 to any given URL)

Do you still think there's a bug?

[kmowery]

No, I completely missed the fact that set_handle_refresh was available from Browser objects.

Upon further testing, the max redirection count does seem to apply.

So no, there isn't a bug. It just felt like there was, as the page I was viewing had a refresh time of 1200 and the open call didn't return for several hours. I suppose it's just a caveat to be aware of: the default settings of mechanize can cause arbitrarily long delays when viewing a page with certain attributes.