Loaders for LSDB and hipscat (reimplementation using LSDB) data

[dougbrn]

Change Description

Resolves #360. Adds an Ensemble loader function for LSDB catalog objects. Reimplements Ensemble.from_hipscat to use LSDB under the hood. Adds a new tutorial to the docs, potentially addressing #367

• [x] My PR includes a link to the issue that I am addressing

Solution Description

The main implementation for both functions is fairly straightforward, just piping the LSDB catalog dataframes into Ensemble.from_dask_dataframe. There's a few things that contributed to API/implementation complexity:

• Ensemble.from_hipscat needs the object_index and source_index kwargs in addition to the id_col specified in the column mapper. These are needed to do a join between object and source to reseat the source index on top of the object level _hipscat_index. I tried to add information to the docstring to inform on how these should be used.
• Support for no object catalog; in keeping with TAPE tradition, I wanted to keep the ability to load in from just a source table. This adds some complication to the implementation, as you should not use the _hipscat_index in this case (it's unique per source). And if you don't use the _hipscat_index, you now almost certainly need to sort. I tried to add a few warnings to catch some problematic inputs, and even added a try except to fall back to using sort=True in cases where the user provided incorrect sort/sorted kwargs. My rationale being that it's better for user experience to warn them of a kwarg mistake but still yield a valid result.
• Ability to choose an index other than _hipscat_index. When you load both an object and source catalog, I think it's pretty much always correct (?) to use the _hipscat_index. We should be pre-sorted by it, and it's guaranteed to be in both catalogs. However a user could choose something else in the column mapper API. I provided a longer warning in these cases about what this other id column will need, but opted not to force the user into using the _hipscat_index (although I'm still definitely considering it).

Code Quality

• [x] My code builds (or compiles) cleanly without any errors or warnings
• [x] My code contains relevant comments and necessary documentation

Project-Specific Pull Request Checklists

• [ ] I have added a function that requires a sync_tables command, and have added the neccesary sync_tables call

Bug Fix Checklist

• [ ] My fix includes a new test that breaks as a result of the bug (if possible)
• [ ] My change includes a breaking change
  • [ ] My change includes backwards compatibility and deprecation warnings (if possible)

New Feature Checklist

• [x] I have added or updated the docstrings associated with my feature using the NumPy docstring format
• [x] I have updated the tutorial to highlight my new feature (if appropriate)
• [x] I have added unit/End-to-End (E2E) test cases to cover my new feature
• [ ] My change includes a breaking change
  • [ ] My change includes backwards compatibility and deprecation warnings (if possible)

Documentation Change Checklist

• [ ] Any updated docstrings use the NumPy docstring format

Build/CI Change Checklist

• [ ] If required or optional dependencies have changed (including version numbers), I have updated the README to reflect this
• [ ] If this is a new CI setup, I have added the associated badge to the README

Other Change Checklist

• [ ] Any new or updated docstrings use the NumPy docstring format.
• [ ] I have updated the tutorial to highlight my new feature (if appropriate)
• [ ] I have added unit/End-to-End (E2E) test cases to cover any changes
• [ ] My change includes a breaking change
  • [ ] My change includes backwards compatibility and deprecation warnings (if possible)

[github-actions[bot]]

Before [65d1fbfa]	After [63b48a6f]	Ratio	Benchmark (Parameter)
41.6±0.1ms	42.8±0.3ms	1.03	benchmarks.time_prune_sync_workflow
39.5±0.2ms	40.1±0.2ms	1.02	benchmarks.time_batch

Click here to view all benchmarks.

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (65d1fbf) 95.27% compared to head (f26917d) 95.40%.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #374 +/- ## ========================================== + Coverage 95.27% 95.40% +0.13% ========================================== Files 25 25 Lines 1671 1698 +27 ========================================== + Hits 1592 1620 +28 + Misses 79 78 -1 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[review-notebook-app[bot]]

Check out this pull request on 

See visual diffs & provide feedback on Jupyter Notebooks.

---

Powered by ReviewNB