maxgio92 / krawler

A crawler for kernel releases distributed by the major Linux distributions.
https://krawler.dev
Mozilla Public License 2.0
13 stars 0 forks source link

Introduce parallelism #20

Closed maxgio92 closed 2 years ago

maxgio92 commented 2 years ago

Motivation

Crawling with no parallelism is time intensive.

Feature

Introduce parallelism on work that can run without executing critical regions, so without concurrency.

Additional context

Kernel releases are crawled from kernel source / headers packages. For each Linux distribution there are plenty of package repositories, and for each repositories potential plenty of packages. In turn, each package could need to be inspected (read) to retrieve additional informations (e.g. compiler version).