LSSTDESC / skyCatalogs

Create sky catalogs and provide access via API
BSD 3-Clause "New" or "Revised" License
6 stars 4 forks source link

U/jrbogart/parquet star2 #82

Closed JoanneBogart closed 7 months ago

JoanneBogart commented 7 months ago

Add ability to make star catalogs from the new parquet files (as well as old-style sqlite file).This is essentially the same code as was used to make star catalogs for the elias region but with starting point = current main. Note that it will not do the right thing in general because it only looks at input parquet files sufficient for the elias region, a quick and dirty approach for the joint Roman-Rubin simulation. In the future I plan to rewrite the parquet files, partitioning them in a more healpix-friendly fashion. It will then be easy to create star skyCatalogs for any region.

JoanneBogart commented 7 months ago

it seems like it would be much better to pass either the path to the sqlite db file or the directory containing the parquet files as an argument to the create_sc.py script instead of the format being used

Perhaps. Passing long file paths as arguments is awkward on the command line. If I did this for stars, it would be nice to do the same for galaxies (cosmodc2 or diffsky), but there is no difference in physical format for galaxies. One would have to parse the GCRCatalogs name. There are a number of additional items one might want to pass to create_sc, but the interface is already rather cumbersome, mitigated somewhat by the ability to use options files. These issues should be addressed in a separate PR.