Lambda value for the penalty

[otuns]

Hello,

Thank you for bringing deepsurv to R.

I have a question, please. In python, Ridge, lasso, and elastic net are the penalty functions used for the model and you choose the one you want. For example, using ridge regression, you can change the lambda value yourself after doing hyperparameter search. I'm not sure about the penalty function you used in the R package and also, how do we change the lambda value for the penalty we want to use?

[RaphaelS1]

Hi @otuns this is a great question because I've looked through the pycox code and also can't figure out the answer but I want to know! Can you send me an example of what you mean "In Python" then I can help you out?

In the theory DeepSurv uses L2 (ridge)

[otuns]

Thank you for your reply.

What I am trying to say is this: Deepsurv package in python has L2_reg: coefficient for L2 weight decay regularization. This is used to help prevent the model from overfitting. It also has L1_reg: coefficient for L1 weight decay regularization. If we set L1_reg to zero, then we are only dealing with ridge . While running the model in python, we need to tune/change the value of L2_reg(for ridge) to get the best CI. The loss function from the code looks like this:

    loss = (
        self._negative_log_likelihood(self.E, deterministic)
        + regularize_layer_params(self.network,l1) * L1_reg
        + regularize_layer_params(self.network, l2) * L2_reg
    )

and where to tune the L2_reg is this: hyperparams = { 'L2_reg': 12.0, 'batch_norm': True, 'dropout': 0.4, 'hidden_layers_sizes': [25, 25], 'learning_rate': 1e-05, 'lr_decay': 0.001, 'momentum': 0.9, 'n_in': X_train_data['x'].shape[1], 'standardize': True }

From the above, I used 12 for my L2_reg .

My question is, for deepsurv in R, can we tune the L2_reg value? If yes, where and how are we going to do that?

Thank you

[RaphaelS1]

Thanks, this is very helpful. The code in survivalmodels uses code from the Python package pycox and not the package Deepsurv. As far as I can see, there is no way to set the l2_reg in the DeepSurv model in pycox.

If native R packages (e.g. torch/keras) prove as fast as the reticulate/Python method, then I'll likely just rewrite everything in R once I get the chance, it will let me address questions like this more satisfactorily.

In the short-term, it might be best to write an issue in the pycox package

[otuns]

Thank you so much.