Feedback for linear regression

[iglee]

feedback:

• add input/output data in the description, problem statement
• reconsider performance metrics r2/mse- what would be a reasonable performance to expect for these models? how to baseline these performances? (refer to LangRank paper)
• consider comparing predicting metrics from each data point vs. predicting mean value of the metric
• correlation per bucketed features?
• double check data processing
• try other models like xgboost

leftover still to do:

• other dataset features yet to be added: word overlap
• basis expansion- code on local, but not organized

beyond predicting bleu/mover_score/etc. from uriel/input data/sys output/reports, consider following analysis:

• system by system analysis: for which systems/buckets did one system do better/worse than expected
• what are the features of the language that are correlated with over/under-performance on particular phenomenon
• system performs better for one metric vs. another? metric vs. metric analysis

[iglee]

Another thing that might be helpful is if you reported the results in a somewhat structure format for each type of experiment, something like: What are the inputs (e.g. “uriel features, distance features, or both”)? What are the things that you’re predicting (e.g. “fine-grained evaluation scores”)? What was the result you would expect from the experiment a-priori? (e.g. “predictions would be best with both, second best with distance features, and third-best with uriel features”) What is the result that you actually got and what is your interpretation (e.g. “results with both are worse than the other two, maybe the model is overfitting”)?