openzim / openedx

Open edX (to zim) scraper
GNU General Public License v3.0
8 stars 7 forks source link

Basic multithreading support #144

Closed satyamtg closed 4 years ago

satyamtg commented 4 years ago

This allows parallel downloading of the different xblock contents and hence reduces the offlining time. Though the downloads done inside the xblocks are not parallelized, this should anyways give a performance boost as there are a large number of xblocks in any course.

The following changes are made -

satyamtg commented 4 years ago

I disagree with the removal of youtube-dl update. This will lead to issues in the future. Also, checked the zimfarm and we want to keep single thread for youtube (will update scraperlib as well). Finally, we're missing a changelog entry

Did those changes.