Quality of information of the logs when submission failed due to incorrect prediction

[lionel-spinelli]

Hello,

I am testing the behavior of the submission process against error in the produced data.

1. I have modified the lines returning the result in the "program" function so that the output is a simple vector of 15 values instead of the required table of 30 columns.

  cut_prop = prop[1:15]
  return(cut_prop)

The submission go in failed (normal) but the error log is not very clear and will not help understand the source of the error. Here is the error log:

Error in Aest[rownames(Aref), ] : no 'dimnames' attribute for array
Calls: scoring_function
Execution halted

2. Then I have modified the lines returning the result in the "program" function so that the output is a table of 15 columns instead of 30 columns.

  cut_prop =prop[ ,1:15]
  return(cut_prop)

here again the error log is not very clear and will not help understand the source of the error. Here is the error log:

Error in M1 - M2 : non-conformable arrays
Calls: scoring_function -> eval_MAE -> MAE -> mean
Execution halted

Would it be possible to improve the log information so the participant will not loose too many time when having errors on their submission data ?

[nicomy]

Hello Lionel,

Thank you for your tests and propositions. I have added, with the commit 4e48b86, the function validate_pred <- function(pred, nb_samples , nb_cells, col_names) to both the submission_script.R and scoring.R.

This function returns a warning if:

• The law of proportion is not respected #12
• The prediction matrix does not match the row names of the ground truth.

And returns an error if:

• The prediction does not have the correct dimension

More tests can be added in the future. Ty