dlnnlsn
commented
3 years ago I think that the normal way that we think of test weights of course doesn't make sense in the Magic Method context. Usually, we take a weighted sum (or average) of all of the test scores. If test j has weight w_j, then student i's overall score is sum_j (wj a{ij}). Now the whole point of the Magic Method is to interpolate scores where they are not available, so we would then replace a_{ij} with s_i d_j, making the student's overall score sum_j (w_j d_j s_i) = (sum_j w_j d_j) s_i. But (sum_j w_j d_j) is just a constant, so ranking students according to this total is then equivalent to ranking them according to the ideal score s_i produced by the Magic Method, so weights in the traditional sense are meaningless.
I think that what Herman's intuition when we say something like "50% is allocated to Stellenbosch Senior tests, 30% to monthlies" is that Senior tests are 5/3 as important as monthlies. If we actually go by 50% as a category for Senior tests that is then divided equally among the tests, then each Stellenbosch test becomes 1/3 as important as the monthlies as a whole. In this case to capture our intuition about what weights mean, I think that every Stellenbosch test should be weighted 50, and every monthly should be weighted 30, so that each Stellenbosch test is 5/3 as important as each given monthly.
On the other hand, if "50% is allocated to Stellenbosch Senior tests, 30% to monthlies" should mean that the total for the Stellenbosch tests is 5/3 as important as the total for the monthlies, then I think that the reasonable interpretation is to divide the 50% among the Stellenbosch tests and weight each one 10%. Though in this case, if we have 3 monthlies that count (approximately like usual), then this division would make each individual Stellenbosch test as important as each individual monthly.
So after chatting with Herman about the weights for Stellenbosch, April, and IMO selection, it seems like we're going to go with a scheme wherein, taking the April Camp selection as an example, 50% is allocated to Stellenbosch Senior tests, 30% to monthlies and 20% to the SAMO 3rd round. So I think we should have each test/source of scores being allocated to a category, and the weights specified on a category level, with each category weight divided equally among the tests in that category to calculate the test weights which are used in the objective function.