search

twVolc / PyCamPermanent

Permanent PiCam (SO2) installation project software
GNU General Public License v3.0
2 stars 2 forks source link

r-squared may be wrong #90

Closed twVolc closed 2 months ago

twVolc commented 10 months ago

I'm pretty sure there's a problem with how r-squared is being calculated. It can produce negative numbers, which should be impossible. They aren't seen on the plot below because the axis is fixed between 0-1, but the saved values do drop below 0. r-squared image

twVolc commented 10 months ago

Note this is on really bad data - usually R-squared does look reasonable, but there still must be a slight issue

ubdbra001 commented 10 months ago

Here is where R-squared is calculated: https://github.com/twVolc/PyCamPermanent/blob/4edfb0ca0b651f07c21951ce248f6114d62a5060/pycam/so2_camera_processor.py#L3909-L3916

This means that (mse / np.var(self.calib_pears.cd_vec)) produces a value that is greater than 1, which either means that mse is very big or the variablity of the cd_vec is very small. It would be good to know what the R-squared and mse values are for a few points when it goes negative.

tpering commented 2 months ago

So I have done some looking into this and it is possible to get a negative R2 (based on how it is calculated here). It is an indication that the model fit is very very very poor (for maths reasons), we can trust the R2 calculation.