astronomy-commons / hipscat-import

HiPSCat import - generate HiPSCat-partitioned catalogs
https://hipscat-import.readthedocs.io
BSD 3-Clause "New" or "Revised" License
5 stars 3 forks source link

Run margin generation within the main import pipeline #384

Open hombit opened 3 weeks ago

hombit commented 3 weeks ago

Feature request

I believe that margin generation should become more implicit and happen by default for every hipscat-import. I propose falling changes:

  1. Generate margins with default 10'' margin_threshold within the "main" importing pipeline. This should be the default, however we still may allow a separate margin generation pipeline to a) generate more caches, b) split the pipeline for large/complicated imports.
  2. Turn off fine filtering by default

Before submitting Please check the following: