seabbs
commented
1 month ago Delay distribution connoisseur @zsusswein mentioned his interest in a delay-level model rather than individual-level model. I think something along these lines would be a good candidate. Reasoning: we have a slow but correct/good model, let's include a fast but less correct/good model. Naturally I think then we could perform some comparisons.
I'm strongly opposed to doing this as part of 0.1.0 or indeed anything other than its own work package. For the reason that it:
- Will be quite a bit of work
- Requires validation/science
- Has lots of choices that need to be discussed.
I agree that a early stage additional model would be helpful here. I think we could proposed a few things here.
- Use a naive direct pass through to
brms. We originally discounted this as duplicating functionality available elsewhere and because we would never recommend users make use of it. However I now think this was a mistake because:
- @athowes has needed to run default
brmsmodels to test understanding of different families and them being available in a different interface has drastically slowed this down. - I think this makes it much easier for users to check for themselves if the methods in this package are "worthwhile". Since we previously talked about this I have moved to think this is more important.
- For both our development work and for users being able to see how the default brms model works within the package should be a helpful education about how all the other models work (i.e without all of their other complexity). This feels like it would be very useful for us right now and of lesser but still use to users on an ongoing basis.
- Implement the dynamic adjusted model we used in the paper. This has some nice scaling benefits and is a sensible choice in a few settings (in particular meta-analysis where you don't have observation time data). The downside is that implementing this model will be more complex than the straight default brms option and so take more time (which I think would be better spent getting the package in a usable state with 1 recommended model). If we pick to do 1. I think we should do this model before any others because we have already done the evaluation/science side of the work.
delay-level model rather than individual-level model. I think something along these lines would be a good candidate. Reasoning: we have a slow but correct/good model, let's include a fast but less correct/good model. Naturally I think then we could perform some comparisons.
I also don't think this is the correct reasoning about the trade-offs here. We can have a cohort based event time model is fast and an individual delay model that is slow (and vice versa). The fact we are already confused to me is a red flag that we need more package in place before looking at this (in particularly functionality for data preprocessing and mapping between possible model formats).
If there is a operational need for CFA to have a cohort based model in the near term I would suggest making an issue to decouple the reference model in epinowcast as this will give the desired properties (as will using say estimate_truncation from EpiNow2 more quickly than we can likely manage here).
athowes
I think that it would be beneficial for us to start thinking about including another model in the package beyond
latent_individual. Aside from providing additional functionality, I think it'd be good sooner rather than later to test how our design decisions hold up to a new model.Delay distribution connoisseur @zsusswein mentioned his interest in a delay-level model rather than individual-level model. I think something along these lines would be a good candidate. Reasoning: we have a slow but correct/good model, let's include a fast but less correct/good model. Naturally I think then we could perform some comparisons.
@seabbs @kgostic @parksw3 what are your opinions about this? Is there a model from the paper (and paper code) that you think would be best to have a go at reimplementing with our current infrastructure? Let me know and I can have a go.