search

SzymonNowakowski / DMRnet

This is a development version of DMRnet — Delete or Merge Regressors Algorithms for Linear and Logistic Model Selection and High-Dimensional Data.
1 stars 0 forks source link

Problems in synthetic simulations #39

Closed SzymonNowakowski closed 1 year ago

SzymonNowakowski commented 2 years ago

There were problems with SS matrix in simulate.R from this private repository in commit 2c6cd3c3ff17c739e7b9dcb59070b1b9beb685ba. The .txt files were generated from the original commit seed, while .2.txt files were generated from seed+1. All in all it seems that the seed value is not that important, but for some reason settings 4..6 have problems with snr=0.66..1.5, rho=0 with the SS matrix in cross validation. result_6_1_1.txt result_6_2_1.2.txt result_6_2_1.txt result_6_3_1.txt result_4_2_1.2.txt result_4_2_1.txt result_5_1_1.2.txt result_5_1_1.txt result_6_1_1.2.txt

SzymonNowakowski commented 2 years ago

I reproduced 8 cases out of 9 from this issue in commit 806fb45 in hard_case_DMRnet_simulations.R script. The 9th case, from result_4_2_1.txt file seems to have been derived for seed+1 too because the run from seed+0 didn't fail for setting 4, 2nd snr and rho=0.

SzymonNowakowski commented 2 years ago

Apparently fixed in 0.3.2.9002.

SzymonNowakowski commented 1 year ago

There still remains two warnings in 0.3.3: see the relevant tag

[1] ">>>>> FAMILY:"
[1] "binomial"
[1] ">>>>> INDEXATION:"
[1] "dimension"
[1] ">>>>> ALGO:"
[1] "DMRnet"
[1] "5_1_1 seed+0, DMRnet"
[1] "5_1_1 seed+1, DMRnet"
[1] "6_1_1 seed+0, DMRnet"
[1] "6_1_1 seed+1, DMRnet"
[1] "6_2_1 seed+0, DMRnet (the fastest error happening in the full model)"
[1] "6_2_1 seed+1, DMRnet"
[1] "6_3_1 seed+0, DMRnet"
[1] "4_2_1 seed+1, DMRnet"
Warning message:
In graphics::arrows(1, min(stats::na.omit(x$cvm)), 1, y, length = 0.05,  :
  zero-length arrow is of indeterminate angle and so skipped
[1] ">>>>> FAMILY:"
[1] "binomial"
[1] ">>>>> INDEXATION:"
[1] "GIC"
[1] ">>>>> ALGO:"
[1] "DMRnet"
[1] "5_1_1 seed+0, DMRnet"
[1] "5_1_1 seed+1, DMRnet"
[1] "6_1_1 seed+0, DMRnet"
[1] "6_1_1 seed+1, DMRnet"
[1] "6_2_1 seed+0, DMRnet (the fastest error happening in the full model)"
[1] "6_2_1 seed+1, DMRnet"
[1] "6_3_1 seed+0, DMRnet"
[1] "4_2_1 seed+1, DMRnet"
[1] ">>>>> FAMILY:"
[1] "binomial"
[1] ">>>>> INDEXATION:"
[1] "dimension"
[1] ">>>>> ALGO:"
[1] "glamer"
[1] "5_1_1 seed+0, DMRnet"
[1] "5_1_1 seed+1, DMRnet"
[1] "6_1_1 seed+0, DMRnet"
[1] "6_1_1 seed+1, DMRnet"
[1] "6_2_1 seed+0, DMRnet (the fastest error happening in the full model)"
[1] "6_2_1 seed+1, DMRnet"
[1] "6_3_1 seed+0, DMRnet"
[1] "4_2_1 seed+1, DMRnet"
Warning message:
In graphics::arrows(1, min(stats::na.omit(x$cvm)), 1, y, length = 0.05,  :
  zero-length arrow is of indeterminate angle and so skipped
SzymonNowakowski commented 1 year ago

Problems with cv plots for sd=0 (zero-length arrow) are apparently fixed in 0.3.4.9002