astronomy-commons / hipscat-import

HiPSCat import - generate HiPSCat-partitioned catalogs
https://hipscat-import.readthedocs.io
BSD 3-Clause "New" or "Revised" License
5 stars 2 forks source link

Gaia margin cache catalog on Epyc has a wrong index dtype and misses its name #319

Closed hombit closed 1 month ago

hombit commented 1 month ago

Bug report

Gaia margin catalog on epyc has a weird metadata for the index - it has name of __index_level_0__ (vs _hipscat_index for "normal" catalogs) and dtype of int64 (vs uint64)

with fsspec.open('https://epyc.astro.washington.edu/~lincc-frameworks/hipscat_surveys/gaia_dr3/gaia_10arcs/_common_metadata') as f:
    meta = pa.parquet.read_metadata(f)
meta.schema
...
  optional int64 field_id=-1 __index_level_0__;

Before submitting Please check the following:

hombit commented 1 month ago

Related to https://github.com/astronomy-commons/lsdb/issues/335