LSSTDESC / DC2-production

Configuration, production, validation specifications and tools for the DC2 Data Set.
BSD 3-Clause "New" or "Revised" License
11 stars 7 forks source link

dpdd related scripts need to be adapted for CFHT/Megacam #360

Closed boutigny closed 3 years ago

boutigny commented 4 years ago

The dpdd related scripts are not compatible with CFHT/Megacam. I have created the branch u/boutigny/cfht to implement the necessary modifications

boutigny commented 4 years ago

I am wondering if we could just keep the filter names as is without changing them to 'u g, r, i, z, y' ? For CFHT, they have u, g, r, i, z, i2, u3, g3, r3, i3, z3 and in some cases we may have fields containing visits with different instances of the same band (for instance, i and i2). So I think that the simplest thing is to just load the filter names from the registry and keep them as is. I have implemented this change to merge_tract_cat.py in the branch u/boutigny/cfht. We may further simplify the code by replacing the filter dictionary with a simple list.

boutigny commented 4 years ago

Unfortunately, trim_tract_cat.py is also assuming some specific filter names. This looks difficult to correct as gcr-catalogs is assuming that the band name is coded with one single character. I think that it would be more general to determine the band names from the files themselves and to pass an actual list of bands to gcr-catalogs methods instead of a string of characters ("ugrizy")

yymao commented 4 years ago
  1. Both merge_tract_cat.py and trim_tract_cat.py are now deprecated, and we should use make_object_catalog.py and merge_parquet_files.py to create object catalogs (cf. instructions)

  2. In make_object_catalog.py (line 99), a filters argument is available. The filters argument only needs to be an iterable of strings, not necessarily single characters. Hence, you can supply filters='u,g,r,i,z,i2,u3,g3,r3,i3,z3'.split(',') and it should work.

  3. This filters argument mentioned above is however not yet exposed as a command line argument of the script, and this is the part needs fixing.

  4. The new object catalog reader in gcr-catalogs also no longer assumes filter names when the object catalogs are generated by make_object_catalog.py.

boutigny commented 4 years ago

Great ! Thanks a lot. I will try that asap

yymao commented 3 years ago

Fixed by #379