feat: support 4bit PQ on new IVF_PQ

[BubbleCal]

also introduces distance_all methods to distance calculator, to improve the new IVF_PQ search performance we store the 4bit PQ codes in half-bytes

[codecov-commenter]

Codecov Report

Attention: Patch coverage is 81.00000% with 38 lines in your changes missing coverage. Please review.

Project coverage is 77.94%. Comparing base (71f323a) to head (27631ee).

Files with missing lines	Patch %	Lines
rust/lance-index/src/vector/sq/storage.rs	0.00%	26 Missing :warning:
rust/lance-index/src/vector/pq/storage.rs	82.35%	11 Missing and 1 partial :warning:

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #3144 +/- ## ========================================== + Coverage 77.93% 77.94% +0.01% ========================================== Files 242 242 Lines 81736 81904 +168 Branches 81736 81904 +168 ========================================== + Hits 63698 63840 +142 - Misses 14849 14891 +42 + Partials 3189 3173 -16 ``` | [Flag](https://app.codecov.io/gh/lancedb/lance/pull/3144/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lancedb) | Coverage Δ | | |---|---|---| | [unittests](https://app.codecov.io/gh/lancedb/lance/pull/3144/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lancedb) | `77.94% <81.00%> (+0.01%)` | :arrow_up: | Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=lancedb#carryforward-flags-in-the-pull-request-comment) to find out more.

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

---

🚨 Try these New Features:

• Flaky Tests Detection - Detect and resolve failed and flaky tests

[BubbleCal]

just added a ticket to track all potential bottlenecks