Adding some sort of callback?

[la55u]

These free proxies we parse from the web are often unreliable and therefore result in an unsuccessful request. I think it would be nice to have some kind of callback to handle this situation. Like on_success() and on_failure() or something.. It would make it asynchronous which would also eliminate the (sometimes very) long waiting time before we get a response from the proxy. I found a possible way to do this without any additional library. I'm thinking about something like this: https://stackoverflow.com/a/44917020 So to be clear, there are two things here i'm suggesting. 1) to make it async and 2) to have an optional callback when the request returns or fails. What do you think? How should this be implemented?

[pgaref]

Hello @la55u

Thats a great idea. Keep in mind that currently:

• RequestProxy has a configurable timeout variable
• When there is an actual timeout the proxy is removed from the list

What we could do for starters is create an async version of the generate_proxied_request method which receives the callback as an argument (similar to your example). Then users can actually switch between sync/async requests on the spot.
Idealy we would reuse all the exception handling and logging code from the generate_proxied_request method and just have two different version of the request call (a sync and an async one).

Does this make sense? Let me know if you want me to open more tickets to break this down into subtasks.