Pull Request: Add Optional Timeout Parameters to HTTP Requests
Summary
This pull request introduces optional timeout parameters to all functions in the FirecrawlApp class that make HTTP requests. This enhancement improves the robustness and resilience of the application by ensuring that calls do not hang indefinitely, which helps prevent resource starvation and supports better error handling.
Key Changes
Added timeout as an optional argument to the following functions:
scrape_url
crawl_url
async_crawl_url
check_crawl_status
....
Updated internal _post_request and _get_request methods to use the timeout value when provided.
Default behavior remains unchanged if timeout is not specified, ensuring backward compatibility.
Benefits
Prevents indefinite blocking on slow or unresponsive network requests.
Enhances system reliability and resilience by managing resource utilization more effectively.
Offers users the flexibility to customize timeout settings based on their specific needs.
Pull Request: Add Optional Timeout Parameters to HTTP Requests
Summary
This pull request introduces optional
timeoutparameters to all functions in theFirecrawlAppclass that make HTTP requests. This enhancement improves the robustness and resilience of the application by ensuring that calls do not hang indefinitely, which helps prevent resource starvation and supports better error handling.Key Changes
Added
timeoutas an optional argument to the following functions:scrape_urlcrawl_urlasync_crawl_urlcheck_crawl_statusUpdated internal
_post_requestand_get_requestmethods to use thetimeoutvalue when provided.Default behavior remains unchanged if
timeoutis not specified, ensuring backward compatibility.Benefits