Parallelize what can be parallelized

smartflutist661 / rfantasy-bingo-stats

Cleanup script(s) for stats.

MIT License

0 stars 0 forks source link

Parallelize what can be parallelized #53

Open smartflutist661 opened 4 months ago

smartflutist661 commented 4 months ago

Run fuzzy matching in the background a. Make sure to lock shared mutables and ensure no duplicate work is performed
Speed up loading by distributing overlap checks?

smartflutist661 commented 3 months ago

None of the work is currently necessarily independent. Would need to ensure there's no overlap between possible matches for item_to_process, including any manually-entered corrections.

Perhaps pipe to collect matches from subprocesses to main, then send a cancel signal to subprocesses if needed?