are NCV penalties working correctly?

[tabpeter]

get latest version

devtools::install_github("pbreheny/biglasso")

> Skipping install of 'biglasso' from a github remote, the SHA1 (d0d0ec27) has not changed since last install.

> Use force = TRUE to force installation

library(biglasso)

> Loading required package: bigmemory

> Loading required package: Matrix

> Loading required package: ncvreg

library(ncvreg) library(glmnet)

> Loaded glmnet 4.1-8

library(reprex)

data(colon) X <- colon$X |> ncvreg::std() X <- cbind(1, X) xtx <- apply(X, 2, crossprod) init <- rep(0, ncol(X)) # cold starts - use more iterations (default is 1000) y <- colon$y og_resid <- resid <- drop(y - X %*% init) og_X <- X.bm <- as.big.matrix(X)

fit1b <- biglasso_fit(X.bm, y, lambda = 0.05, xtx=xtx, r = resid, penalty = "MCP", max.iter = 10000)

fit2b <- ncvfit(X = X, y = y, lambda = 0.05, xtx = xtx, r = resid, penalty = "MCP", max.iter = 10000)

these run fine...

tinytest::expect_equivalent(fit1b$resid, fit2b$resid, tolerance = 0.1)

> ----- PASSED : <-->

> call| tinytest::expect_equivalent(fit1b$resid, fit2b$resid, tolerance = 0.1)

tinytest::expect_equivalent(fit1b$beta, fit2b$beta, tolerance = 0.1)

> ----- PASSED : <-->

> call| tinytest::expect_equivalent(fit1b$beta, fit2b$beta, tolerance = 0.1)

...but the tests below fail -- does this signal a problem?

tinytest::expect_equivalent(fit1b$resid, fit2b$resid, tolerance = 0.01)

> ----- FAILED[data]: <-->

> call| tinytest::expect_equivalent(fit1b$resid, fit2b$resid, tolerance = 0.01)

> diff| Mean relative difference: 0.2640539

tinytest::expect_equivalent(fit1b$beta, fit2b$beta, tolerance = 0.01)

> ----- FAILED[data]: <-->

> call| tinytest::expect_equivalent(fit1b$beta, fit2b$beta, tolerance = 0.01)

> diff| Mean relative difference: 0.2418661

Created on 2024-04-20 with reprex v2.1.0

[pbreheny]

I'm not sure if this is the only issue or not, but here you've written yourself a note to add MCP and SCAD and then you never did so:

https://github.com/pbreheny/biglasso/blob/d0d0ec275cd2e1b10646e38859e369f601a3a174/src/gaussian_simple.cpp#L345-L349

[tabpeter]

Thank you for finding this error - fixing this does solve the problem. Tests now pass at tol = 0.01.

Will get this all resolved and pushed in the next few minutes. From there, biglasso should be ready for CRAN...

---

From: Patrick Breheny @.> Sent: Saturday, April 20, 2024 1:01 PM To: pbreheny/biglasso @.> Cc: Peter, Tabitha K @.>; Author @.> Subject: [External] Re: [pbreheny/biglasso] are NCV penalties working correctly? (Issue #55)

I'm not sure if this is the only issue or not, but here you've written yourself a note to add MCP and SCAD and then you never did so:

https://github.com/pbreheny/biglasso/blob/d0d0ec275cd2e1b10646e38859e369f601a3a174/src/gaussian_simple.cpp#L345-L349

— Reply to this email directly, view it on GitHubhttps://github.com/pbreheny/biglasso/issues/55#issuecomment-2067743630, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AUPYOIGABIS6YAWBFJEMRHTY6KUOXAVCNFSM6AAAAABGQRWQKSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANRXG42DGNRTGA. You are receiving this because you authored the thread.Message ID: @.***>