athowes / multi-agyw

Spatio-temporal estimates of HIV risk group proportions for AGYW across 13 priority countries in sub-Saharan Africa
https://athowes.github.io/multi-agyw
MIT License
2 stars 1 forks source link

Improve runtime of Model 2x and 4x #135

Closed athowes closed 2 years ago

athowes commented 2 years ago

MWI with S = 10 takes:

Begin fitting Model 2x.
Completed fitting Model 2x.
Begin post-processing
Completed post-processing
> end_time <- Sys.time()
> end_time - start_time
Time difference of 14.50404 secs

Next step is to try alternative replicate based approach.

(For reference, Model 1 took 8.9 seconds)

Version with replicate:

Begin fitting Model 2x.
Completed fitting Model 2x.
Begin post-processing
Completed post-processing
> end_time <- Sys.time()
> end_time - start_time
Time difference of 9.482835 secs
athowes commented 2 years ago

Took this long for fewer_countries = TRUE:

Begin fitting Model 2x.
Completed fitting Model 2x.
Begin post-processing
Completed post-processing
> end_time <- Sys.time()
> end_time - start_time
Time difference of 15.31516 mins

Next step is to check how long e.g. Model 1 takes then scale the 15 minutes up (it won't be linear but should give some idea).

Begin fitting Model 1.
Completed fitting Model 1.
Begin post-processing
Completed post-processing
> 
> end_time <- Sys.time()
> end_time - start_time
Time difference of 1.405516 mins

So 10x more than Model 1. I think that with all the data, Model 1:4 take around 30 minutes to fit, which would mean that Model 2x would take at least 5 hours... not good news but at least faster than before.

For some reason 3x is a lot quicker than 2x:

Begin fitting Model 3x.
Completed fitting Model 3x.
Begin post-processing
Completed post-processing
> end_time <- Sys.time()
> end_time - start_time
Time difference of 6.301961 mins

And here is 4x. So I think what we've learned here is that it's the Besag part which is slowing things down. Is there any way to change that?

Begin fitting Model 4x.
Completed fitting Model 4x.
Begin post-processing
Completed post-processing
> end_time <- Sys.time()
> end_time - start_time
Time difference of 36.86065 mins