At the moment, the rjaf() function returns: a) the optimal treatment assignment rule k* for each unit in the validation data (based on X_val alone), and b) the outcome estimates under optimal assignment (let's call them Y_pred).
When we have the oracle data (with the ground truth outcomes), Ypred is the actual ground truth outcome under optimal assignment (using Y{val,k*}). For evaluation of how good the algorithm does, we can simply average out the Y_preds to get the average outcome under optimal assignment.
When we don't have the oracle data (but say data from a randomized trial, where you don't observe the counterfactual outcomes for each unit), Y_pred is outcome estimates based on the forest fitted on the estimation data. Evaluation of the algorithm then needs to use the (observed) Y from the validation data, and we need to discuss if we should have a function in the package which does that.
At the moment, the rjaf() function returns: a) the optimal treatment assignment rule k* for each unit in the validation data (based on X_val alone), and b) the outcome estimates under optimal assignment (let's call them Y_pred).
When we have the oracle data (with the ground truth outcomes), Ypred is the actual ground truth outcome under optimal assignment (using Y{val,k*}). For evaluation of how good the algorithm does, we can simply average out the Y_preds to get the average outcome under optimal assignment.
When we don't have the oracle data (but say data from a randomized trial, where you don't observe the counterfactual outcomes for each unit), Y_pred is outcome estimates based on the forest fitted on the estimation data. Evaluation of the algorithm then needs to use the (observed) Y from the validation data, and we need to discuss if we should have a function in the package which does that.