search

LSSTDESC / snmachine

Machine learning code for photometric supernova classification
BSD 3-Clause "New" or "Revised" License
9 stars 5 forks source link

Fix implementation of pca_svd including reconstruction. #70

Closed cnsetzer closed 4 years ago

cnsetzer commented 5 years ago

At present the components returned from pca_svd catastrophically fail to reconstruct the light curve models that the GPs return. This has been demonstrated for both a general set of objects from the training data by Catarina and myself. There is a notebook which demonstrates the issue. I will link a notebook in the comments below. This seems either to be an issue with pca_svd itself or an issue with how fit_sn uses the wavelet_features, eigenvectors, and means to reconstruct the light curve model.

cnsetzer commented 5 years ago

It appears rather that the output of pca_SVD is not compatible with the reconstruction of the model done using fit_sn. The original eigendecomposition PCA returns variables in the format needed by fit_sn by way of set_model. We should make these outputs match. The pull request merging svd with HierGP was possibly premature.

We did not diagnose this bug primarily due to the pipeline defaulting to read from files that had components previously computed using the old PCA method. It would be good to make this more transparent to the user. We cannot reproduce the pipeline completing all the way to log-loss scores. At the moment it only works up to the PCA decomposition, failing at reconstruction of the lightcurves from the PCA.

rbiswas4 commented 5 years ago

@cnsetzer Could you please link the notebook you mentioned? I think trying to read previously stored components and mixing and matching with the _pca method I added (both for eigenvalue decomposition and svd) will be hard. If we can be convinced that the new method does satisfactorily then we should make a clean break and add this to fit_sn, and recompute things.

cnsetzer commented 5 years ago

@rbiswas4 Yes apologies, I think this might have been an issue due to user-error/late in the day hacking. The notebook should be 24_ExamplePipeline_Ias_only. I will look at this later this afternoon or tomorrow morning, but I think this amounted to misunderstanding due to my limited use of snmachine recently and limited documentation of the code.

Catarina-Alves commented 4 years ago

Solved by #219.