perf: Avoid using `RefCell` for nth index caches

[codecov[bot]]

Codecov Report

All modified and coverable lines are covered by tests :white_check_mark:

Comparison is base (1aadaff) 85.48% compared to head (232c9e1) 85.54%.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## master #282 +/- ## ========================================== + Coverage 85.48% 85.54% +0.05% ========================================== Files 18 18 Lines 1729 1736 +7 ========================================== + Hits 1478 1485 +7 Misses 251 251 ```

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

[codspeed-hq[bot]]

CodSpeed Performance Report

Merging #282 will not alter performance

_{Comparing dd/drop-refcell (232c9e1) with master (1aadaff)}

Summary

✅ 6 untouched benchmarks