As a maintainer, I aim to enhance the Prowlarr scraping workflow to ensure data is accurately scraped with fault tolerance, thereby minimizing data loss when the Prowlarr API is unresponsive. The current implementation doings IMDb search and title search simultaneously, often failing to retrieve data within 10 seconds for both. If it fails, it does not attempt another search until the next Prowlarr search interval is reached.
Solution Idea
Rather than the current workflow, we can perform the live search using the IMDb ID and execute the title search in the background with a fail-retry mechanism to efficiently scrape the data.
Description
As a maintainer, I aim to enhance the Prowlarr scraping workflow to ensure data is accurately scraped with fault tolerance, thereby minimizing data loss when the Prowlarr API is unresponsive. The current implementation doings IMDb search and title search simultaneously, often failing to retrieve data within 10 seconds for both. If it fails, it does not attempt another search until the next Prowlarr search interval is reached.
Solution Idea
Rather than the current workflow, we can perform the live search using the IMDb ID and execute the title search in the background with a fail-retry mechanism to efficiently scrape the data.