Preprocess X and Y for training

LSSTDESC / derp

A first attempt at a simple LSST DRP catalog emulator

BSD 3-Clause "New" or "Revised" License

1 stars 1 forks source link

Closed jiwoncpark closed 6 years ago

jiwoncpark commented 6 years ago

Eventually we'll need to further process X and Y so that

X does not contain object_id
NaN values in X and Y are filled with a token value, like -9999
The positions of filled NaN values in Y are presaved as a mask, so that they are not counted toward the loss (we shouldn't penalize the algorithm for not getting answers that weren't there)

This can be an internal helper method _preprocess_for_training that is called by make_training_sets.

drphilmarshall commented 6 years ago

Yes! All good points. Let's do this in the issue/2/munging branch and consider #2 an epic. No merging of #2 until this issues is resolved!