selective-inference / R-software

R software for selective inference
GNU General Public License v2.0
34 stars 18 forks source link

larInf() crashes when p > n, in fact crashes on processing the nth variable #20

Open DesmondCampbell opened 7 years ago

DesmondCampbell commented 7 years ago

Hello,

I think I have found a bug in the larInf() function of the selectiveInference R package

I think whenever p > n, i.e the number of regressors exceeds the number of rows of data, a singular matrix is generated causeing the function to crash.

My Dataset consists of matrix 360 rows of 777 columns (regressors), plus a response vector (360 long) There are no missing values. All variables have been mean centred. Regressors have been scaled to sd =1.

I fitted a Least Angle Regression via lar() I then ran larInf() with verbose set to TRUE.

The function crashed while processing the 360th variable, Note 360 = number of rows of data.

After this happenned the first time, I dropped one of each pair of some highly correlated regressors (remaining max pearson corr < 0.95). So I don't think it is colinearity.

I would appreciate you updating me about this issue. Also in the interim have any workarounds - LAR, LASSO preferred. For instance perhaps I can only run the model for first 50 steps.

My email is - desmond.campbell@glasgow.ac.uk

Below is R console output fom a crash of larInf() Feel free to contact me.

regards Desmond

fit the lar model

lfit <- NA; lfit = lar( mX, vY, intercept=F, normalize=F )

estimate sigma

sigmaEst <- estimateSigma( mX, vY, intercept=F, standardize=F)

attempt inference - function crashes

out2 = larInf( lfit, sigma=sigmaEst$sigmahat, verbose=T ) Inference for variable 4 ... Inference for variable 3 ... Inference for variable 326 ... Inference for variable 257 ... Inference for variable 9 ... Inference for variable 514 ... Inference for variable 761 ... Inference for variable 615 ... Inference for variable 128 ... Inference for variable 261 ... Inference for variable 332 ... Inference for variable 367 ... Inference for variable 405 ... Inference for variable 200 ... Inference for variable 436 ... Inference for variable 76 ... Inference for variable 118 ... Inference for variable 131 ... Inference for variable 563 ... Inference for variable 215 ... Inference for variable 459 ... Inference for variable 32 ... Inference for variable 371 ... Inference for variable 23 ... Inference for variable 104 ... Inference for variable 235 ... Inference for variable 530 ... Inference for variable 269 ... Inference for variable 365 ... Inference for variable 567 ... Inference for variable 122 ... Inference for variable 57 ... Inference for variable 154 ... Inference for variable 575 ... Inference for variable 479 ... Inference for variable 90 ... Inference for variable 389 ... Inference for variable 355 ... Inference for variable 364 ... Inference for variable 531 ... Inference for variable 291 ... Inference for variable 167 ... Inference for variable 525 ... Inference for variable 303 ... Inference for variable 708 ... Inference for variable 47 ... Inference for variable 502 ... Inference for variable 647 ... Inference for variable 388 ... Inference for variable 538 ... Inference for variable 465 ... Inference for variable 334 ... Inference for variable 166 ... Inference for variable 276 ... Inference for variable 554 ... Inference for variable 685 ... Inference for variable 98 ... Inference for variable 441 ... Inference for variable 73 ... Inference for variable 372 ... Inference for variable 100 ... Inference for variable 420 ... Inference for variable 74 ... Inference for variable 703 ... Inference for variable 178 ... Inference for variable 581 ... Inference for variable 362 ... Inference for variable 608 ... Inference for variable 252 ... Inference for variable 22 ... Inference for variable 751 ... Inference for variable 337 ... Inference for variable 605 ... Inference for variable 729 ... Inference for variable 305 ... Inference for variable 244 ... Inference for variable 678 ... Inference for variable 353 ... Inference for variable 361 ... Inference for variable 251 ... Inference for variable 135 ... Inference for variable 410 ... Inference for variable 638 ... Inference for variable 296 ... Inference for variable 358 ... Inference for variable 182 ... Inference for variable 127 ... Inference for variable 472 ... Inference for variable 527 ... Inference for variable 657 ... Inference for variable 129 ... Inference for variable 193 ... Inference for variable 546 ... Inference for variable 60 ... Inference for variable 10 ... Inference for variable 584 ... Inference for variable 137 ... Inference for variable 328 ... Inference for variable 560 ... Inference for variable 173 ... Inference for variable 550 ... Inference for variable 569 ... Inference for variable 705 ... Inference for variable 242 ... Inference for variable 342 ... Inference for variable 380 ... Inference for variable 161 ... Inference for variable 281 ... Inference for variable 99 ... Inference for variable 754 ... Inference for variable 517 ... Inference for variable 485 ... Inference for variable 65 ... Inference for variable 333 ... Inference for variable 512 ... Inference for variable 39 ... Inference for variable 738 ... Inference for variable 201 ... Inference for variable 412 ... Inference for variable 460 ... Inference for variable 205 ... Inference for variable 645 ... Inference for variable 684 ... Inference for variable 210 ... Inference for variable 591 ... Inference for variable 16 ... Inference for variable 376 ... Inference for variable 458 ... Inference for variable 534 ... Inference for variable 427 ... Inference for variable 561 ... Inference for variable 603 ... Inference for variable 214 ... Inference for variable 660 ... Inference for variable 294 ... Inference for variable 198 ... Inference for variable 765 ... Inference for variable 177 ... Inference for variable 559 ... Inference for variable 558 ... Inference for variable 194 ... Inference for variable 516 ... Inference for variable 454 ... Inference for variable 528 ... Inference for variable 510 ... Inference for variable 524 ... Inference for variable 725 ... Inference for variable 70 ... Inference for variable 707 ... Inference for variable 35 ... Inference for variable 508 ... Inference for variable 249 ... Inference for variable 505 ... Inference for variable 111 ... Inference for variable 770 ... Inference for variable 625 ... Inference for variable 286 ... Inference for variable 483 ... Inference for variable 44 ... Inference for variable 393 ... Inference for variable 686 ... Inference for variable 595 ... Inference for variable 463 ... Inference for variable 26 ... Inference for variable 219 ... Inference for variable 113 ... Inference for variable 320 ... Inference for variable 504 ... Inference for variable 501 ... Inference for variable 335 ... Inference for variable 85 ... Inference for variable 350 ... Inference for variable 395 ... Inference for variable 345 ... Inference for variable 132 ... Inference for variable 442 ... Inference for variable 180 ... Inference for variable 428 ... Inference for variable 639 ... Inference for variable 496 ... Inference for variable 227 ... Inference for variable 315 ... Inference for variable 375 ... Inference for variable 43 ... Inference for variable 641 ... Inference for variable 302 ... Inference for variable 430 ... Inference for variable 622 ... Inference for variable 283 ... Inference for variable 665 ... Inference for variable 452 ... Inference for variable 226 ... Inference for variable 179 ... Inference for variable 7 ... Inference for variable 256 ... Inference for variable 1 ... Inference for variable 243 ... Inference for variable 343 ... Inference for variable 196 ... Inference for variable 148 ... Inference for variable 68 ... Inference for variable 31 ... Inference for variable 153 ... Inference for variable 149 ... Inference for variable 456 ... Inference for variable 440 ... Inference for variable 521 ... Inference for variable 186 ... Inference for variable 566 ... Inference for variable 493 ... Inference for variable 327 ... Inference for variable 139 ... Inference for variable 247 ... Inference for variable 692 ... Inference for variable 447 ... Inference for variable 433 ... Inference for variable 648 ... Inference for variable 360 ... Inference for variable 752 ... Inference for variable 188 ... Inference for variable 494 ... Inference for variable 240 ... Inference for variable 381 ... Inference for variable 613 ... Inference for variable 115 ... Inference for variable 136 ... Inference for variable 378 ... Inference for variable 145 ... Inference for variable 318 ... Inference for variable 397 ... Inference for variable 109 ... Inference for variable 650 ... Inference for variable 369 ... Inference for variable 598 ... Inference for variable 611 ... Inference for variable 217 ... Inference for variable 346 ... Inference for variable 570 ... Inference for variable 475 ... Inference for variable 174 ... Inference for variable 399 ... Inference for variable 114 ... Inference for variable 606 ... Inference for variable 652 ... Inference for variable 125 ... Inference for variable 163 ... Inference for variable 30 ... Inference for variable 719 ... Inference for variable 564 ... Inference for variable 348 ... Inference for variable 351 ... Inference for variable 312 ... Inference for variable 577 ... Inference for variable 107 ... Inference for variable 396 ... Inference for variable 704 ... Inference for variable 726 ... Inference for variable 498 ... Inference for variable 414 ... Inference for variable 551 ... Inference for variable 189 ... Inference for variable 117 ... Inference for variable 264 ... Inference for variable 423 ... Inference for variable 698 ... Inference for variable 199 ... Inference for variable 46 ... Inference for variable 341 ... Inference for variable 744 ... Inference for variable 415 ... Inference for variable 110 ... Inference for variable 124 ... Inference for variable 402 ... Inference for variable 208 ... Inference for variable 609 ... Inference for variable 669 ... Inference for variable 767 ... Inference for variable 400 ... Inference for variable 724 ... Inference for variable 155 ... Inference for variable 319 ... Inference for variable 297 ... Inference for variable 324 ... Inference for variable 599 ... Inference for variable 720 ... Inference for variable 519 ... Inference for variable 555 ... Inference for variable 96 ... Inference for variable 520 ... Inference for variable 583 ... Inference for variable 716 ... Inference for variable 431 ... Inference for variable 384 ... Inference for variable 211 ... Inference for variable 706 ... Inference for variable 424 ... Inference for variable 274 ... Inference for variable 593 ... Inference for variable 42 ... Inference for variable 29 ... Inference for variable 322 ... Inference for variable 172 ... Inference for variable 518 ... Inference for variable 25 ... Inference for variable 299 ... Inference for variable 349 ... Inference for variable 668 ... Inference for variable 644 ... Inference for variable 582 ... Inference for variable 497 ... Inference for variable 681 ... Inference for variable 536 ... Inference for variable 202 ... Inference for variable 548 ... Inference for variable 277 ... Inference for variable 92 ... Inference for variable 717 ... Inference for variable 727 ... Inference for variable 289 ... Inference for variable 688 ... Inference for variable 48 ... Inference for variable 356 ... Inference for variable 713 ... Inference for variable 461 ... Inference for variable 731 ... Inference for variable 133 ... Inference for variable 218 ... Inference for variable 221 ... Inference for variable 547 ... Inference for variable 612 ... Inference for variable 522 ... Inference for variable 263 ... Inference for variable 556 ... Inference for variable 279 ... Inference for variable 579 ... Inference for variable 116 ... Inference for variable 562 ... Inference for variable 448 ... Inference for variable 568 ... Inference for variable 63 ... Inference for variable 486 ... Inference for variable 682 ... Inference for variable 231 ... Inference for variable 119 ... Inference for variable 492 ... Inference for variable 499 ... Inference for variable 191 ... Inference for variable 457 ... Inference for variable 680 ... Inference for variable 590 ... Inference for variable 578 ... Inference for variable 278 ... Inference for variable 614 ... Inference for variable 737 ... Inference for variable 157 ... Inference for variable 241 ... Inference for variable 753 ... Inference for variable 316 ... Inference for variable 37 ... Inference for variable 207 ... Error in solve.default(crossprod(x[, c(A, j), drop = F]), c(sA, sj)) : system is computationally singular: reciprocal condition number = 2.65433e-19

Enter a frame number, or 0 to exit

1: larInf(lfit, sigma = sigmaEst$sigmahat, verbose = T) 2: covtest.pval(obj, sigma, j) 3: solve(crossprod(x[, c(A, j), drop = F]), c(sA, sj)) 4: solve.default(crossprod(x[, c(A, j), drop = F]), c(sA, sj))

Enter an item from the menu, or 0 to exit Selection: 2 Called from: covtest.pval(obj, sigma, j) Browse[1]> str(obj) List of 19 $ lambda : num [1:360] 802 706 666 620 610 ... $ action : num [1:360] 4 3 326 257 9 514 761 615 128 261 ... $ sign : num [1:360] 1 1 1 -1 1 -1 1 -1 1 1 ... $ df : num [1:360] 0 1 2 3 4 5 6 7 8 9 ... $ beta : num [1:777, 1:360] 0 0 0 0 0 0 0 0 0 0 ... ..- attr(, "dimnames")=List of 2 .. ..$ : NULL .. ..$ : chr [1:360] "801.501" "706.463" "666.388" "619.644" ... $ completepath: logi TRUE $ bls : num [1:777] 7.02 0 -41.17 50.22 0 ... $ Gamma : num [1:430199, 1:360] -1.24 -1.34 -2.24 -1.91 -2.02 ... $ nk : num [1:360] 1553 3105 4655 6203 7749 ... $ vreg : num [1:360, 1:360] -3.41e-03 -2.25e-03 3.01e-05 9.73e-04 6.04e-04 ... $ mp : num [1:360] 706 666 620 610 561 ... $ x : num [1:360, 1:777] -0.0126 0.7447 1.3939 0.5283 0.2038 ... ..- attr(, "dimnames")=List of 2 .. ..$ : NULL .. ..$ : chr [1:777] "age" "bmi" "iSexMale" "iRaceWhite" ... $ y : num [1:360] -12.22 11.78 -4.22 -2.22 -3.22 ... $ bx : num [1:777] 0 0 0 0 0 0 0 0 0 0 ... $ by : num 0 $ sx : num [1:777] 1 1 1 1 1 1 1 1 1 1 ... $ intercept : logi FALSE $ normalize : logi FALSE $ call : language lar(x = mX, y = vY, intercept = F, normalize = F)

Enter a frame number, or 0 to exit

1: larInf(lfit, sigma = sigmaEst$sigmahat, verbose = T) 2: covtest.pval(obj, sigma, j) 3: solve(crossprod(x[, c(A, j), drop = F]), c(sA, sj)) 4: solve.default(crossprod(x[, c(A, j), drop = F]), c(sA, sj))

Selection: 4 Browse[4]> str(A) Error during wrapup: object 'A' not found Browse[4]> str(sA) Error during wrapup: object 'sA' not found Browse[4]> sA Error during wrapup: object 'sA' not found Browse[4]> sj Error during wrapup: object 'sj' not found Browse[4]>

Enter a frame number, or 0 to exit

1: larInf(lfit, sigma = sigmaEst$sigmahat, verbose = T) 2: covtest.pval(obj, sigma, j) 3: solve(crossprod(x[, c(A, j), drop = F]), c(sA, sj)) 4: solve.default(crossprod(x[, c(A, j), drop = F]), c(sA, sj))

Selection: 3 Called from: solve(crossprod(x[, c(A, j), drop = F]), c(sA, sj)) Browse[3]> x [,1] [,2] [,3] [,4] [,5] [,6] [,7] [,8] [,9] [,10] [1,] -0.135924521 -0.53959038 1.19214893 -0.01421069 -1.041016541 0.326034267 0.26500638 -0.960023700 -0.51561099 -1.20493243 [2,] -0.040796970 -1.50080114 -0.12313786 0.13012750 -0.431907053 -0.398122516 1.38694989 -0.751053458 0.39568325 -2.07117872 [3,] 1.010539007 0.26789815 -0.57416484 0.18401390 -0.777816318 -0.395627017 -0.29149014 -0.936544143 -0.78301461 1.46689919 [4,] -0.158262445 0.03422677 -1.49031686 0.14435823 0.373681784 -1.159683959 -1.83027672 -1.493705888 0.88538861 -1.81137952 [5,] -2.156637503 -0.99527307 2.36382893 -0.58794149 0.312564581 -1.852945215 -1.62766978 -0.313418720 -0.38212891 -0.74986617 [6,] 0.498646835 -0.77030057 -2.95103621 0.38704784 0.071228643 -0.005530756 -0.01630346 -1.384058435 0.05333596 -0.43174933 [7,] -0.755244306 -0.57808092 0.16527818 -1.90768563 0.678440355 1.461921770 1.00335522 0.166683520 -0.55154057 -0.48930007 [8,] 0.778585191 -0.93296826 -0.83427349 0.54545178 -2.439386238 -1.279680283 1.09715982 0.394343957 -1.20007454 1.11291377 [9,] 0.754577954 -2.17699470 0.49191538 0.38998743 -0.309635615 0.333371417 -0.66300952 -1.513257952 0.32198376 1.64118771 [10,] -1.099545613 -0.51411027 -0.31836686 -0.93410691 0.679744123 -0.086414090 0.96810912 2.341575314 1.65470449 1.92244707 [11,] 0.167349961 0.08911827 2.00514227 -1.58162900 1.014932965 1.045619649 -1.68767450 -1.317586126 0.43220392 -0.16430612 [12,] -0.029283481 0.29449012 -0.42367291 0.08779191 1.538640017 1.207992289 -0.46206989 -0.482461635 -1.14869873 -0.69258761 [13,] 1.875845028 0.69219193 -0.21993684 0.61936957 -1.766977094 0.481993148 -0.75647305 1.164762118 0.84888363 -0.49852718 [14,] 0.244634467 -1.61165588 1.04970016 1.25773169 2.289596615 0.017207862 -0.36149330 -2.220113051 -2.09969534 0.75007988 [15,] 0.702153686 0.18790080 -0.13176235 -0.07475668 -3.031246490 0.195700113 -0.12897901 -0.760771901 0.63486366 0.59624472 [16,] -0.015085665 0.07739601 0.34877761 -1.37696708 1.697693381 -0.532431684 -1.90806483 0.113902749 1.33088703 -0.19282119 [17,] -0.143370080 -0.17381374 -0.33805463 2.58334531 1.716225099 -1.747889162 -1.33484260 0.307544658 1.31283814 -0.25875486 [18,] 0.320666578 -1.39938990 0.90388559 -0.34682734 0.632786360 0.776295312 -1.97481963 1.695464227 -0.21660778 -0.49068080 [19,] 0.122334080 -0.36917683 0.84678400 -0.68131382 -0.617648033 -1.011039297 -0.57179915 0.471684001 -0.34040911 0.85024124 [20,] -0.594665192 0.45122022 1.15815376 -0.15466483 0.819283675 -0.170147151 -0.43803850 -0.091859204 0.89581581 -0.37425964 [21,] -0.442394755 1.62599840 2.68699190 -1.05947874 -1.317948105 0.243743597 -1.08256786 1.239914587 -0.21308478 1.14866509 [22,] 0.290503585 1.92329780 0.49434109 -0.13113782 1.581622553 -0.049417265 1.34234670 0.665728403 1.03842731 -1.62377562 [23,] 0.723653379 -0.16308618 -0.06669678 -0.29460770 0.542882170 -0.488798968 -0.18147054 0.648302852 -1.20255449 0.30756340 [24,] 0.459573950 -0.43878404 0.45234730 -1.36093856 -1.124450875 -0.895214637 -0.72844405 0.509580868 -0.34449943 -1.93496164 [25,] 0.185105160 1.19149970 0.58797623 -0.72952160 -0.174150393 0.207755767 0.97336677 -1.271143267 0.40021085 -0.11758417 [26,] 0.233805005 1.66993913 2.35390936 -0.30308914 0.830663022 -0.803728605 0.28198861 -0.382541835 -0.68465177 0.50432419 [27,] 0.593338262 1.13319174 0.97410755 1.28481802 1.050625407 1.816677838 -1.00219170 0.641766036 1.11519590 0.66781248 [28,] 2.001166926 -1.51748309 3.47986354 0.25304566 -0.873195032 1.307126285 -2.17789210 0.809061912 -0.09256017 -0.49002282 [29,] -1.837197758 2.73116619 0.02218368 0.79971121 0.414821577 2.220612436 1.04578863 0.077064865 0.13630547 0.24127207 [30,] -0.862390941 0.49927385 0.22159352 -0.92523637 0.496264134 0.895634975 -1.09656178 -0.987741449 -0.20264900 -0.62741259 [31,] 1.582887396 -1.49192711 -1.10596642 -1.84019303 0.010476067 1.763370740 0.34219906 -0.617517489 0.05698383 -2.11222684 [32,] 0.155200689 0.92173907 -1.09526404 0.08390022 -0.069542105 -0.916548375 -0.91594460 -1.184800346 -0.68427970 -0.19521214 [33,] -0.275019336 -0.51671477 0.63095093 1.56749078 -1.358048321 0.574185163 -0.33869176 0.529320119 0.62316739 0.11255449 [34,] 0.787675758 2.10530277 -0.48174932 1.46943133 -0.718815197 0.414021504 0.53115492 0.620213159 1.49887045 -0.21075028 [35,] -0.222988089 -0.88697108 -0.50296748 -0.65801536 0.398364733 -2.253295882 -0.18876424 0.024531029 -0.81106140 0.05212177 [36,] 1.392023729 -0.16780993 0.44485833 -0.35756506 -1.503152999 -0.495622571 -0.36220336 -0.791666008 2.09897861 0.18689630 [37,] -0.488652910 0.90935102 -0.16208895 0.24534311 -0.006880948 0.117982213 0.26067733 0.349656408 -2.12973949 0.70279824 [38,] 0.137377602 -1.39461339 0.58773983 -1.50030833 0.192743969 0.768607719 0.35868153 0.723573485 -0.62370678 -0.23986249 [39,] 0.003508096 0.81374561 -0.33266209 -0.87705486 -1.718899822 0.371091953 0.09259231 -0.719597432 1.11478469 0.08818384 [40,] -0.726885772 -0.95730687 0.53980555 -0.91290569 -0.519845254 0.527382255 -0.64299329 1.742491823 0.51877834 1.64375486 [41,] -0.721432863 -0.36983033 0.19829531 1.11951450 -0.720731915 0.046013913 0.90831952 1.151706980 0.67539050 0.20732545 [42,] -0.190524592 1.47883722 -0.05916185 -0.74054927 -0.861262991 0.713252252 0.91436715 0.408624974 -0.83930861 0.05546456 [43,] 1.335229412 -0.60414215 1.80066109 0.37009155 0.703867683 1.070627575 -0.66908482 -0.791556143 -1.01869590 -1.50168851 [44,] 0.355995577 -0.29060017 0.24280906 0.35524485 0.483979524 0.941098404 0.72646992 -0.735756883 0.17971169 0.09992694 [45,] 0.842759563 -1.78442144 -1.01239335 1.83961657 0.268628353 -0.603922419 -0.50300216 -1.854904995 1.36773691 -1.20195591 [46,] 0.774532574 2.25806515 -0.03640304 -0.46668736 0.268219343 -0.417661088 -0.22235378 -0.253015529 -0.58629073 0.27760027 [47,] 0.080459477 -0.66697896 1.22664690 0.84007939 -1.814923955 -0.505385186 0.33449550 1.277028155 0.58206897 -0.43474862 [48,] -0.672642017 0.47793931 0.77365964 0.10818225 0.789873389 -0.687525294 0.04592375 -0.157158147 -0.98304011 0.50841786 [49,] 1.835522538 -1.16107581 -0.93221744 -0.94851528 1.134906928 -0.145932374 -1.93770334 0.334439067 0.01211304 0.83270525 [50,] -0.207207497 -0.38027931 0.65490057 -0.07096093 0.760204081 -0.675620176 -1.29488628 -0.008979237 -0.34374124 -0.78807061 Browse[3]> c(A, j) Error during wrapup: object 'A' not found Browse[3]> c(sA, sj) Error during wrapup: object 'sA' not found Browse[3]>

Enter a frame number, or 0 to exit

1: larInf(lfit, sigma = sigmaEst$sigmahat, verbose = T) 2: covtest.pval(obj, sigma, j) 3: solve(crossprod(x[, c(A, j), drop = F]), c(sA, sj)) 4: solve.default(crossprod(x[, c(A, j), drop = F]), c(sA, sj))

Selection: 0

covtest.pval Error: object 'covtest.pval' not found No suitable frames for recover()

this is the fitted LAR model

str(lfit) List of 19 $ lambda : num [1:360] 802 706 666 620 610 ... $ action : num [1:360] 4 3 326 257 9 514 761 615 128 261 ... $ sign : num [1:360] 1 1 1 -1 1 -1 1 -1 1 1 ... $ df : num [1:360] 0 1 2 3 4 5 6 7 8 9 ... $ beta : num [1:777, 1:360] 0 0 0 0 0 0 0 0 0 0 ... ..- attr(, "dimnames")=List of 2 .. ..$ : NULL .. ..$ : chr [1:360] "801.501" "706.463" "666.388" "619.644" ... $ completepath: logi TRUE $ bls : num [1:777] 7.02 0 -41.17 50.22 0 ... $ Gamma : num [1:430199, 1:360] -1.24 -1.34 -2.24 -1.91 -2.02 ... $ nk : num [1:360] 1553 3105 4655 6203 7749 ... $ vreg : num [1:360, 1:360] -3.41e-03 -2.25e-03 3.01e-05 9.73e-04 6.04e-04 ... $ mp : num [1:360] 706 666 620 610 561 ... $ x : num [1:360, 1:777] -0.0126 0.7447 1.3939 0.5283 0.2038 ... ..- attr(, "dimnames")=List of 2 .. ..$ : NULL .. ..$ : chr [1:777] "age" "bmi" "iSexMale" "iRaceWhite" ... $ y : num [1:360] -12.22 11.78 -4.22 -2.22 -3.22 ... $ bx : num [1:777] 0 0 0 0 0 0 0 0 0 0 ... $ by : num 0 $ sx : num [1:777] 1 1 1 1 1 1 1 1 1 1 ... $ intercept : logi FALSE $ normalize : logi FALSE $ call : language lar(x = mX, y = vY, intercept = F, normalize = F)

  • attr(*, "class")= chr "lar"

my dataset

str(vY) num [1:360] -12.22 11.78 -4.22 -2.22 -3.22 ...

str(mX) num [1:360, 1:777] -0.0126 0.7447 1.3939 0.5283 0.2038 ...

  • attr(*, "dimnames")=List of 2 ..$ : NULL ..$ : chr [1:777] "age" "bmi" "iSexMale" "iRaceWhite" ...

fnSS(vY) variableId n n.obs n.na n.uniq min max mean median var sd mad skewness dfData dfData 360 360 0 44 -24.21667 27.78333 -2.021091e-16 0.7833333 66.15348 8.133479 7.413 -0.1404218

range(apply(mX,2,mean)) [1] -4.179720e-17 3.701858e-17 range(apply(mX,2,sd)) [1] 1 1

session info

sessionInfo() R version 3.3.2 (2016-10-31) Platform: x86_64-w64-mingw32/x64 (64-bit) Running under: Windows 7 x64 (build 7601) Service Pack 1

locale: [1] LC_COLLATE=English_United Kingdom.1252 LC_CTYPE=English_United Kingdom.1252 LC_MONETARY=English_United Kingdom.1252 LC_NUMERIC=C LC_TIME=English_United Kingdom.1252

attached base packages: [1] parallel stats graphics grDevices utils datasets methods base

other attached packages: [1] selectiveInference_1.2.2 intervals_0.15.1 covTest_1.02 MASS_7.3-45 glmpath_0.97 survival_2.40-1 lars_1.2 glmnet_2.0-5 Matrix_1.2-8
[10] visreg_2.3-0 RColorBrewer_1.1-2 tableone_0.8.1 abind_1.4-5 doParallel_1.0.10 iterators_1.0.8 foreach_1.4.3 gplots_3.0.1 tidyr_0.6.1
[19] broom_0.4.1 snowfall_1.84-6.1 snow_0.4-2 dplyr_0.5.0 NMF_0.20.6 Biobase_2.34.0 BiocGenerics_0.20.0 cluster_2.0.5 rngtools_1.2.4
[28] pkgmaker_0.22 registry_0.3 moments_0.14

loaded via a namespace (and not attached): [1] Rcpp_0.12.9 lattice_0.20-34 class_7.3-14 zoo_1.7-14 gtools_3.5.0 assertthat_0.1 digest_0.6.11 psych_1.6.12 gridBase_0.4-7 R6_2.2.0 plyr_1.8.4 survey_3.32-1
[13] e1071_1.6-7 ggplot2_2.2.1 lazyeval_0.2.0 gdata_2.17.0 splines_3.3.2 stringr_1.1.0 foreign_0.8-67 munsell_0.4.3 mnormt_1.5-5 tibble_1.2 codetools_0.2-15 bitops_1.0-6
[25] grid_3.3.2 nlme_3.1-128 xtable_1.8-2 gtable_0.2.0 DBI_0.5-1 magrittr_1.5 scales_0.4.1 KernSmooth_2.23-15 stringi_1.1.2 reshape2_1.4.2 tools_3.3.2 colorspace_1.3-2
[37] caTools_1.17.1

DesmondCampbell commented 7 years ago

Hello again,

I've just found out one of the solutions I suggest stops it crashing. That is, I limit the number of steps taken by lar() to < p For instance, the following line completes successfully lfit <- NA; lfit = lar( mX, vY, intercept=F, normalize=F, maxsteps=100 )

The glmnet() option dfmax may similarly curtail the steps taken. I'm not sure.

regards Desmond