dswah / pyGAM

[HELP REQUESTED] Generalized Additive Models in Python
https://pygam.readthedocs.io
Apache License 2.0
871 stars 159 forks source link

Assessing significance of smooth terms with permutation testing #341

Open avaruuser opened 10 months ago

avaruuser commented 10 months ago

Hello, previously people have investigated the use of permutations for hypothesis testing of the smooth terms in GAMs (e.g. here). I'm relatively new to GAMs and wondered if you see any issues with how I implemented such a permutation testing procedure using pyGAM:

  1. Compute a GAM with the DV and IVs.
  2. Store the deviance explained from the GAM output using gam.score(IVs, DV)
  3. Randomly shuffle one of the IVs n_perms times (e.g. 10,000 times).
  4. Compute the GAM with the randomly shuffled IV.
  5. Store the deviance explained for each random permutation.
  6. Get p-value from the proportion of permutations that exceed the original deviance explained.
  7. Repeat this process for the other IVs

Does this make sense to you?