astronomy-commons / hipscat-import

HiPSCat import - generate HiPSCat-partitioned catalogs
https://hipscat-import.readthedocs.io
BSD 3-Clause "New" or "Revised" License
5 stars 3 forks source link

Add `lowest_order` to import arguments #249

Closed camposandro closed 5 months ago

camposandro commented 5 months ago

Feature request

It would be useful to also compute the pixel map with the lowest_order. We need it in the import pipeline arguments.

https://github.com/astronomy-commons/hipscat-import/blob/ea600421b14e3546dc7f81b31a8017f1560e6963/src/hipscat_import/catalog/run_import.py#L140-L145

Before submitting Please check the following:

nevencaplar commented 5 months ago

@camposandro What is the current behavior, without the lowest order specified?

camposandro commented 5 months ago

@delucchi-cmu Please correct me if I am wrong, but at the moment we can run the importer with either a constant pixel order or a highest pixel order. Specifying a highest order we can control the maximum order the catalog is generated at but not its minimum order (unless we want to play trial-and-error with the value of pixel threshold). E.g. for very small and sparse datasets (like the ones in unit tests), the likelihood that we will obtain a catalog of order 0 / 1 is high. If we want to generate a catalog of at least order 3 it's easier to just use the lowest_order parameter and compute the pixel map accordingly.