Use multiprocessing to speed up processing of input files, using the number of processes which matches the CPU count.
On my old dual-core Mac, the processing step speeds up:
601 files (all of 2021): 2m7s -> 1m20s
2,629 files (my full Strava archive): 10m29s -> 5m9s
Will be a larger gain for machines with more CPUs.
multiprocessing needs a single worker function, this is the new process_file which decides which of process_gpx and process_fit to call. They also need to be top-level functions to work with multiprocessing.
Use
multiprocessing
to speed up processing of input files, using the number of processes which matches the CPU count.On my old dual-core Mac, the processing step speeds up:
Will be a larger gain for machines with more CPUs.
multiprocessing
needs a single worker function, this is the newprocess_file
which decides which ofprocess_gpx
andprocess_fit
to call. They also need to be top-level functions to work withmultiprocessing
.I also formatted the file using the popular Black autoformatter:
I also fixed a bug when using an input path like
~/dir
, it needs to expand~
before checking it's a dir (and appending*
).