Open njawadekar opened 1 year ago
There's no universal guideline, a 50/50 train/test split is just a reasonable default. And for different splits, that's normal considering trees are discontinuous. Ideally, if there is signal, the estimated rewards are similar even though the splits are different: there may be many trees that give a reasonable policy, it's not necessarily unique. Here is a tutorial if that's helpful.
I notice much variation in the variables that are selected by the policytree, depending on the size of the random training subset that is specified, in this step:
Fit a depth 2 tree on a random training subset.
n <- 250
train <- sample(1:n, 200)
opt.tree <- policy_tree(X[train, ], Gamma.matrix[train, ], depth = 2)
opt.tree
Are there recommended best practices regarding the size (or % of the original sample) for the random training subset when setting up the policytree?