Tabbing to Font size ToggleGroupControl applies a font size

Description

When using a keyboard, you can't tab past the Typography Size in the Block Settings Sidebar without also applying a font size. There's no way to get past the field and also remain at the default style (no font size option).

Step-by-step reproduction instructions

Select a paragraph without any font sizes applied
Tab to the block settings sidebar
Continue tabbing past the font size selector in the Typography panel
The first font size option will be applied when you tab to the font size toggle group panel
Tab past the field, and the option will still be applied

Screenshots, screen recording, code snippet

https://github.com/WordPress/gutenberg/assets/967608/5d7eeffb-2df9-46ed-aef5-7e7aff0f15d6

Video shows the cursor on a paragraph in the post editor. Tab keypresses move focus to the Block settings sidebar for the paragraph. There is no option selected for the Typography Font Size. When the tab key moves focus to the font size selector, the first option (small) is selected. Tab key is pressed again to move past the field and the small option is still selected, even though it was never intentionally acted on (enter or space keypress).

Environment info

No response

Please confirm that you have searched existing issues in the repo.

Yes

Please confirm that you have tested with all plugins deactivated except Gutenberg.

Yes

WordPress / gutenberg