astronomy-commons / hipscat-import

HiPSCat import - generate HiPSCat-partitioned catalogs
https://hipscat-import.readthedocs.io
BSD 3-Clause "New" or "Revised" License
5 stars 3 forks source link

Confusing error behavior if margin generation results in empty catalog #258

Open delucchi-cmu opened 5 months ago

delucchi-cmu commented 5 months ago

Bug report

If you set a margin that is too low to result in any objects falling in the margins, the pipeline fails on the parquet_metadata.write_parquet_metadata call with varying error messages, depending on your file system.

On a local filesystem:

pyarrow.lib.ArrowInvalid: ParquetDatasetFactory must contain a schema with at least one column

For azure:

FileNotFoundError: abfs://hipscat/pytests/data/small_sky_order1_margin

If there are no points found in the first mapping step, then we should error out of the pipeline early, stating that no margin objects were found and they should increase the margin.

Before submitting Please check the following: