MusicConnectionMachine / StructuredData

In this project we will be scanning structured online resources such as DBPedia, Worldcat, MusicBrainz, IMSLP and other databases
GNU General Public License v3.0
4 stars 3 forks source link

enable threading for scraping scripts #67

Closed TimHenkelmann closed 7 years ago

TimHenkelmann commented 7 years ago

One can now specify the number of threads/cpus that should be used for scraping, i.e. node cli.js -s dbpedia -t 4 will use 4 cpus. If this parameter is omitted, the script will use all available cpus.