Closed knaaptime closed 3 years ago
tagging @darribas and @ljwolf in case there is a better way to do the parallelization. Currently the threading backend is much faster for simulate_evenness
and simulate_systematic_randomization
, but multiprocessing seems to work much better for simulate_reallocation
. The loky backend always crashes for any of them?
cc: @martinfleis
scratch that... the loky backend is fastest for all approaches, i just needed to update joblib and friends
@sjsrey @renanxcortes ok, this should be all set. We get a nice speedup in the inference classes thanks to the parallelization, the only side effect is that setting a random seed is a bit more complicated, so i opted to turn the test precision down to two decimals
Merging #174 (dcae6bb) into master (403bcf6) will decrease coverage by
2.28%
. The diff coverage is61.57%
.
@@ Coverage Diff @@
## master #174 +/- ##
==========================================
- Coverage 65.77% 63.49% -2.29%
==========================================
Files 116 118 +2
Lines 4322 4421 +99
==========================================
- Hits 2843 2807 -36
- Misses 1479 1614 +135
Impacted Files | Coverage Δ | |
---|---|---|
segregation/network/network.py | 25.58% <0.00%> (-1.70%) |
:arrow_down: |
segregation/tests/test_batch_compute.py | 100.00% <ø> (ø) |
|
segregation/tests/test_multiscalar_profile.py | 100.00% <ø> (ø) |
|
segregation/util/network.py | 84.31% <0.00%> (ø) |
|
segregation/inference/randomization.py | 21.62% <21.62%> (ø) |
|
segregation/singlegroup/spatial_dissim.py | 87.17% <50.00%> (ø) |
|
segregation/inference/comparative.py | 61.64% <61.64%> (ø) |
|
segregation/batch/batch_compute.py | 90.36% <75.75%> (-9.64%) |
:arrow_down: |
segregation/inference/inference_wrappers.py | 72.80% <78.43%> (-13.51%) |
:arrow_down: |
segregation/_base.py | 82.47% <80.00%> (-0.69%) |
:arrow_down: |
... and 31 more |
Continue to review full report at Codecov.
Legend - Click here to learn more
Δ = absolute <relative> (impact)
,ø = not affected
,? = missing data
Powered by Codecov. Last update 403bcf6...dcae6bb. Read the comment docs.
excellent, thanks James. I was pretty sure #177 only touched infrastructure, so figured it was easiest to continue focusing on this PR while touches more. I wasnt sure if there would be conflicts and since 177 is smaller it seemed easier to rebase. I'm open to suggestions if you think it would be easier some other way though
excellent, thanks James. I was pretty sure #177 only touched infrastructure, so figured it was easiest to continue focusing on this PR while touches more. I wasnt sure if there would be conflicts and since 177 is smaller it seemed easier to rebase. I'm open to suggestions if you think it would be easier some other way though
Yep, probably better to do #177 after this PR merged.