search

gasparrini / dlnm

R package dlnm
68 stars 13 forks source link

Possible to use shrinkage penalized cubic spline? #5

Closed Aariq closed 3 years ago

Aariq commented 4 years ago

Is it possible to use a shrinkage version of a penalized cubic regression spline in crossbasis()? Something equivalent to bs="cs" in mgcv::s()?

gasparrini commented 4 years ago

Hi Eric,

Thanks for the interest. The development of penalized DLNMs is described in a recent paper: http://www.ag-myresearch.com/2017_gasparrini_biomet.html. The method is fully implemented in R by embedding functions from the dlnm package in mgcv. There are a couple of options:

Follow the link to the GitHub code (https://github.com/gasparrini/2017_gasparrini_Biomet_Rcodedata) that reproduce the results in the paper. The simulations indicated a good performance, although the (rare) applications I tried with real data returned sometimes strange results. If you manage to use the methodology in published works, I would be interested to see them.

Best

Antonio Gasparrini, BSc Mbiol MSc PhD

Professor of Biostatistics and Epidemiology London School of Hygiene & Tropical Medicine 15-17 Tavistock Place, London WC1H 9SH, UK Office: 0044 (0)20 79272406 Mobile: 0044 (0)79 64925523 Skype contact: a.gasparrini Twitter: @AGasparrini75 http://www.ag-myresearch.comhttp://www.ag-myresearch.com/

From: Eric R Scott notifications@github.com Sent: 03 January 2020 22:36 To: gasparrini/dlnm dlnm@noreply.github.com Cc: Subscribed subscribed@noreply.github.com Subject: [gasparrini/dlnm] Possible to use shrinkage penalized cubic spline? (#5)

Is it possible to use a shrinkage version of a penalized cubic regression spline in crossbasis()? Something equivalent to bs="cs" in mgcv::s()?

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHubhttps://github.com/gasparrini/dlnm/issues/5?email_source=notifications&email_token=ADHVOGTAWFLRQJ7Z5JHH5WDQ36VVTA5CNFSM4KCSF4BKYY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4ID6WLJA, or unsubscribehttps://github.com/notifications/unsubscribe-auth/ADHVOGXXDFXAO22L3ISPTG3Q36VVTANCNFSM4KCSF4BA.

Aariq commented 4 years ago

Thank you so much for your quick response and easily reproducible code! I think I'm getting close to a solution. In order to understand the dlnm package and other code you've provided so that I can apply it to my own data, I've been trying to replicate results from Teller et al 2016. In this paper, they use bs="cs" in a GAM which does a specific kind of shrinkage penalty according to the help file:

s(x,bs="cs") specifies a penalized cubic regression spline which has had its penalty modified to shrink towards zero at high enough smoothing parameters (as the smoothing parameter goes to infinity a normal cubic spline tends to a straight line.)

This type of shrinkage is essential for replicating the results in Teller et al. Without it, the coefficients "overshoot" the true value.

Do you have any advice on how to implement this in the dlnm framework? It looks to me like I'll have to create my own penalty matrix, but I would prefer to use the default behavior of bs="cs" if possible.

I can't use the external method because fun = "cs" is not supported by dlnm. I can't simply use s(...bs = "cs") in the internal method, because this basis only handles 1D smooths.

My ideal code would be:

cb_temp <- crossbasis(
  temperature,
  lag=25,
  argvar=list(fun="poly"),
  arglag=list(fun="cs", df=10)
  )
gasparrini commented 4 years ago

Hi Eric,

it was not easy to implement this method, as mgcv has a very clever structure but also a bit cumbersome in terms of extensions. In my opinion, the easiest way would be to modify the code in smooth.construct.cb.smooth.spec.R of dlnm, which basically provides an additional smoother for crossbasis transformations directly used by the function s() of gam(). I suggest using the original code in dlnm_2.3.9.tar.gz from https://cran.r-project.org/web/packages/dlnm/index.html, which also includes the comments. You can see in the code (lines 30-34) that at the moment only 'ps' and 'cr' unidimensional smoothers are called for producing the two-dimensional crossbasis. Maybe it won't be difficult to include an additional basis such as 'cs'.

Good luck with it

Best -Antonio Gasparrini

Aariq commented 3 years ago

I took a look at smooth.construct.cs.smooth.spec and it's simply smooth.construct.cr.smooth.spec with an attribute "shrink" of 0.1. Seems like it might not be too difficult to get the "cs" basis to work with a crossbasis. If I get it to work, I'll make a pull request.