LSSTDESC / DC2-production

Configuration, production, validation specifications and tools for the DC2 Data Set.
BSD 3-Clause "New" or "Revised" License
11 stars 7 forks source link

Issues/310 #314

Closed wmwv closed 5 years ago

wmwv commented 5 years ago

Adds Run 1.2i Object catalogs along with simple scripts to document in detail. Parquet file DPDD files (simple format, one file for all tracts) for Run 1.2i and 1.2p. Documents how they were generated.

yymao commented 5 years ago

To be added as one of the steps (after the step of updating the reader):

We can use the reader to generate the schema files. Simply run:

import GCRCatalogs

for cat in ('dc2_object_run1.2i', 'dc2_object_run1.2i_all_columns'):
    GCRCatalogs.load(cat).generate_schema_yaml()

No further change to the reader is needed. The next time the same catalog is being loaded, the reader will automatically pick up the schema file.

wmwv commented 5 years ago

@yymao I've added a note about GCRCAtalogs.load_catalog(cat).generate_schema_yaml()

But how do the custom defaults get set?

yymao commented 5 years ago

@wmwv right now it is hard coded in https://github.com/LSSTDESC/gcr-catalogs/blob/master/GCRCatalogs/dc2_object.py#L544

wmwv commented 5 years ago

Ah, thanks. I had never quite tracked that down.

wmwv commented 5 years ago

@yymao Are there any other changes you'd like to see to this before merging?

yymao commented 5 years ago

@wmwv One minor comment -- for the Shifter image of DM+DESC environment section, do you want to just suggest the use of start-kernel-cli.py, so that we don't need to update this README when there are future updates to the DESC kernels. The usage is simply

python start-kernel-cli.py desc-stack
wmwv commented 5 years ago

I will mention start-kernel-cli.py here and use it with #315. But it's not actually what I did for regenerating Run 1.2i.