search

LAHTeR / htr-quality-classifier

Detect quality of (digitized) text.
GNU General Public License v3.0
3 stars 0 forks source link

Parallelize processing. #5

Open carschno opened 1 year ago

carschno commented 1 year ago

This does not seem to increase processing speed. On my local machine with 10 CPU cores, speed decreased from ~6 files per second to ~1.5 files per second when using 10 parallel workers.

The bottleneck might be I/O rather than CPU, so parallelization would be redundant altogether.

sonarcloud[bot] commented 1 year ago

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

20.0% 20.0% Coverage
0.0% 0.0% Duplication