[Parallel Downloads in Deezer & Freyr] - LIDARR - Parallel Downloads in Deezer and Freyr

[baxterblk]

Is your feature request related to a problem? Please describe. I'd like to see parallel downloading with Freyr if possible. I run a massive library and grabbing missing items quickly in the background would be nice. I guess most folks would appreciate a faster download process.

Describe the solution you'd like

1. Run download functions in the background to allow parallel processing. This can be managed by sending processes to the background using the & operator.

2. Implement a mechanism to control the number of concurrent downloads, preventing system overload. This can be done using a simple semaphore-like control using Bash's job control features or an array to track active processes.

3. Synchronization Point: After initiating the downloads, a synchronization point will be necessary to ensure all processes have completed before moving on to the next batch or ending the script.

Implement Concurrency Control:

Create a control mechanism to limit the number of concurrent downloads. This could be an array that stores process IDs of the background tasks.

[Example]

max_jobs=5
active_jobs=()

function manage_jobs {
    if [[ ${#active_jobs[@]} -ge $max_jobs ]]; then
        wait -n
    fi
}

function add_job {
    manage_jobs
    "$@" &
    active_jobs+=($!)
}

function cleanup_jobs {
    for job in "${active_jobs[@]}"; do
        wait $job
    done
}

Describe alternatives you've considered Other yt-dlp (LidaTube) alternatives. Even added yt-dlp to this would be a tremendous upgrade.

[RandomNinjaAtk]

Considering these are pulling for online services, I think it would be best to keep it as least resource intensive as possible to avoid unnecessary attention...

Also making the script run things in parallel makes things a bit more complicated to handle overall and would require some re-working...

[RandomNinjaAtk]

I appreciate the suggestion, but I don't think its a good idea based on what I stated previously and would require a decent amount of re-working to make it happen that I just don't have the cycles for....

In the past the scripts have hammered some of the online services too much, which resulted in issues that ultimately resulted in taming them more....