search

LSSTDESC / DC2-production

Configuration, production, validation specifications and tools for the DC2 Data Set.
BSD 3-Clause "New" or "Revised" License
11 stars 7 forks source link

Issues/317 #318

Closed wmwv closed 5 years ago

wmwv commented 5 years ago

Using Pandas instead of AstroPy to join across filters speeds up AFW->HDF5 conversion by a factor of ~8 when testing on one patch (tract: 5066, patch='3,0').

A tract (5066) with 31 filled-in patches took 15 minutes instead of 2-4 hours.

The new method produces an HDF file of the exact same size as the previous now. I'll do explicit validation over the next few days.

wmwv commented 5 years ago

I've compared the catalogs generated by the old and new methods in tract 4850 and confirm that they are identical.