Performance on detecting shift and scale is not very good. This could be for at least three reasons:
a) The synthetic problems are too hard. i.e. Suppose that the random patches that we are generating include scaling by 1% of the mean. It will be difficult for any statistical method to detect this.
b) The cost for the scale and shift operators need to be set differently.
c) ???? Some unknown reason. Both (a) and (b) are fairly likely explanations, so this is less likely.
So I guess what I would do is try a few different costs for the shift and scale operators, e.g., 5 different costs geometrically spaced by factors of 2 (or 10) around the current cost --- just rerunning the shift and scale experiments with those costs. This would allow us to plot a precision recall curve if we want, although for now we could just look at the numbers in the table to see if there is a good setting of precision and recall.
CS wrote on 17/08/2017:
Performance on detecting shift and scale is not very good. This could be for at least three reasons:
a) The synthetic problems are too hard. i.e. Suppose that the random patches that we are generating include scaling by 1% of the mean. It will be difficult for any statistical method to detect this.
b) The cost for the scale and shift operators need to be set differently.
c) ???? Some unknown reason. Both (a) and (b) are fairly likely explanations, so this is less likely.
So I guess what I would do is try a few different costs for the shift and scale operators, e.g., 5 different costs geometrically spaced by factors of 2 (or 10) around the current cost --- just rerunning the shift and scale experiments with those costs. This would allow us to plot a precision recall curve if we want, although for now we could just look at the numbers in the table to see if there is a good setting of precision and recall.