Remove optional dependencies without extras

[staticdev]

Closes #2206

[codecov[bot]]

Codecov Report

Merging #2207 (f7a6b0e) into main (4be9850) will not change coverage. The diff coverage is n/a.

Additional details and impacted files

```diff @@ Coverage Diff @@ ## main #2207 +/- ## ======================================= Coverage 99.15% 99.15% ======================================= Files 39 39 Lines 3090 3090 Branches 748 748 ======================================= Hits 3064 3064 Misses 15 15 Partials 11 11 ```

[miketheman]

Thanks for the quick fix! Is there a plan to release this as 5.13.1 or such soon? The added deps fail our dependency update pipeline, and I'd prefer to not pin to isort prior to 5.13 to unblock it if possible.

[staticdev]

@miketheman done

[miketheman]

Thanks!