secastel / aFC

Calculates allelic Fold Change (aFC) using standard input files for fastQTL.
21 stars 11 forks source link

optimize the bootstrap function. #2

Closed brentp closed 6 years ago

brentp commented 6 years ago

These changes result in a 4-5X speedup over master.

To test, I set the seed (commented out) and ran the original aFC.py with timings and then the code with these changes. The output is identical.

However the timings for this branch are:

6. Calculating eQTL effect sizes...
     COMPLETED 100 of 30694 = 0.003258 in 260 seconds
     COMPLETED 200 of 30694 = 0.006516 in 112 seconds
     COMPLETED 300 of 30694 = 0.009774 in 114 seconds
     COMPLETED 400 of 30694 = 0.013032 in 111 seconds
     COMPLETED 500 of 30694 = 0.016290 in 106 seconds
     COMPLETED 600 of 30694 = 0.019548 in 106 seconds
     ...

while master shows:

6. Calculating eQTL effect sizes...
     COMPLETED 100 of 30694 = 0.003258 in 724 seconds
     COMPLETED 200 of 30694 = 0.006516 in 559 seconds
     COMPLETED 300 of 30694 = 0.009774 in 608 seconds
     ...

There are further optimizations possible, but I think these are the lowest hanging fruit with the smallest changes.

brentp commented 6 years ago

ping @secastel

secastel commented 6 years ago

Sorry for the delay! Had to find the time to do internal testing and make sure the results are the same. Everything looks good, thanks so much for the fixes!

brentp commented 6 years ago

cheers