Closed malramsay64 closed 6 years ago
Merging #44 into master will decrease coverage by
0.17%
. The diff coverage is82.08%
.
@@ Coverage Diff @@
## master #44 +/- ##
==========================================
- Coverage 83.22% 83.04% -0.18%
==========================================
Files 18 19 +1
Lines 1341 1380 +39
==========================================
+ Hits 1116 1146 +30
- Misses 225 234 +9
Impacted Files | Coverage Δ | |
---|---|---|
src/sdanalysis/util.py | 92.85% <100%> (+3.11%) |
:arrow_up: |
src/sdanalysis/main.py | 86.84% <100%> (+0.54%) |
:arrow_up: |
src/sdanalysis/threading.py | 74.07% <74.07%> (ø) |
|
src/sdanalysis/read.py | 93.36% <80%> (-1.37%) |
:arrow_down: |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update 4ddbee9...1f72e3e. Read the comment docs.
This uses the python multiprocessing module to process a collection of files in parallel, while using an alternate thread to write the files. There is no support for parallel writing of HDF5 files and alternate methods don't work, so this is the only method of parallelism which will work given the constraints I am working in.
It turns out that creating parallel threads for processing is rather simple once the appropriate functions are set up.