Open stephens999 opened 4 years ago
I think this is more of a question of providing clear information to the user, rather than finding the right defaaults.
I believe that points of confusion such as these would go away if verbose = TRUE
and susie
provided more information about what it is doing as it fits the model / variational approximation.
For example, estimate_residual_variance = TRUE
as default seems reasonable as long as it is clear to the user. For example, in varbvsmix
the console output looks like this:
variational max. --------- hyperparameters ---------
iter lower bound change sigma mixture sd's mix. weights (drop)
0001 -4.774371e+03 3.0e-01 8.7e+00 [0.1,1] [0.013,0.121] (0)
0002 -3.734355e+03 4.6e-01 6.4e+00 [0.1,1] [0.002,0.281] (0)
0003 -3.065023e+03 5.1e-01 4.1e+00 [0.1,1] [0.000,0.503] (0)
0004 -2.361188e+03 7.0e-01 2.0e+00 [0.1,1] [0.000,0.743] (0)
0005 -1.758400e+03 8.0e-01 1.0e+00 [0.1,1] [0.000,0.896] (0)
0006 -1.514304e+03 4.5e-01 9.1e-01 [0.1,1] [0.000,0.949] (0)
0007 -1.479079e+03 2.5e-01 9.3e-01 [0.1,1] [0.000,0.969] (0)
0008 -1.468320e+03 2.0e-01 9.5e-01 [0.1,1] [0.000,0.980] (0)
0009 -1.463366e+03 1.4e-01 9.6e-01 [0.1,1] [0.000,0.986] (0)
0010 -1.460862e+03 1.0e-01 9.7e-01 [0.1,1] [0.000,0.989] (0)
This shows that sigma
is changing at each iteration.
I was just running and wanted to fix
residual_variance
. But when I specify it, it is not actually fixed (due to default thatestimate_residual_variance=TRUE
).Similar is true for
scaled_prior_variance
: you can specify it but it will still be estimated by default.I think this behavior is confusing.
I am wondering about these possible alternatives:
1a. if user specifies
residual_variance
andestimate_residual_variance
is missing, emit warning that it will be estimated by default. (With similar behavior for prior variance).1b. if user specifies
residual_variance
andestimate_residual_variance
is missing then setestimate_residual_variance = FALSE
and emit warning (With similar behavior for prior variance).residual_variance = "estimate"
(which would also be the default). Here we would remove theestimate_residual_variance
parameter. This would seem clearer with hindsight, but does not allow user to specify initial values for these parameters. (these could be added as separate arguments,residual_variance_init
of course). ... again I think this would be clearer, but be a breaking change to the interface.