search

astronomy-commons / hats

Hierarchical Adaptive Tiling Scheme
https://hats.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
17 stars 5 forks source link

Do we support datasets with no ra and dec columns? #370

Open smcguire-cmu opened 1 month ago

smcguire-cmu commented 1 month ago

hipscat-import I think has pipeline options that support using the index to import with no ra and dec columns, but our catalog_info files have ra and dec columns as required fields. Should we have some better way in metadata to indicate that a catalog doesn't have explicit ra and dec columns?

nevencaplar commented 1 month ago

The imported data does not have ra/dec, but the index would link to a different catalog that does have ra/dec? I believe that we had cases like that. Is that what you imagine? I am not sure that I would understand importing data that would have no spatial information.

smcguire-cmu commented 1 month ago

Yeah, that's the case I'm thinking of, and I think we have tests in hipscat-import that support this. (Although now with the healpix_29 index we'll need to use another column to link to the original catalog.)

The main thing this issue is for is that in the catalog_info file, we have the ra_column and dec_column parameters, which are required to have a value currently. I think when we import a catalog with no ra and dec columns, these are filled with the default column names ra and dec, which is incorrect. So if we want to support catalogs without the columns, these params should probably change to optional and we'll have to check for it everywhere.

nevencaplar commented 1 month ago

Can we enable so that ra and dec get populated from the main table that has ra and dec values?