We should return the test for falsy values, otherwise we risk Astropy autocasting the column into a MaskedColumn without asking us. This is a symptom of the fact that the ButlerStandardizer is version dependent and we have data in our datastore that was processed, exported and then re-ingested, from different stack versions.
When the stack processing the data is of a later version than the one we use with ButlerStandardizer, and has removed one or more properties from objects like visit_info or summary_stats (which we use to extract metadata in the Butler), the returned value is sometimes ambiguous and is interpreted as a MaskedConstant (by either pybind or astropy). This auto-casts the entire column into a masked column which causes unexpected behavior.
The solution is either:
set the appropriate obvious, unambiguous, falsy value in the ButlerStandardizer itself (probably the best)
force the masked columns in the ImageCollection itself into a regular column by calling col.filled(default_value) and replacing the masked column
See https://github.com/dirac-institute/kbmod/pull/695.
We should return the test for falsy values, otherwise we risk Astropy autocasting the column into a MaskedColumn without asking us. This is a symptom of the fact that the ButlerStandardizer is version dependent and we have data in our datastore that was processed, exported and then re-ingested, from different stack versions.
When the stack processing the data is of a later version than the one we use with ButlerStandardizer, and has removed one or more properties from objects like visit_info or summary_stats (which we use to extract metadata in the Butler), the returned value is sometimes ambiguous and is interpreted as a MaskedConstant (by either pybind or astropy). This auto-casts the entire column into a masked column which causes unexpected behavior.
The solution is either:
col.filled(default_value)and replacing the masked column