string-processing f-string debug expressions quotes changed when using conversion

[MeGaGiGaGon]

Describe the bug

When using unstable, f-string quotes get incorrectly changed if the expression contains a conversion. This changes program behavior. As an example, "" f'{""=!r}' currently formats to f"{''=!r}".

>>> print("" f'{""=!r}')
""=''
>>> print(f"{''=!r}")
''=''

To Reproduce

Format this code

"" f'{""=!r}'

Playground link

Expected behavior

The program behavior should be unchanged.

Additional context

I found this while investigating how to fix #4493/#4494. The issue comes in two parts:

• String concatenation allows for f-string quotes to be changed.
  • This is the same issue that leads to both #4493 and #4494.
  • Both still have the same issue even if the first string isn't an f-string.
• The regex used to detect quotes in debug f-string expressions is flawed
  • Here's the current regex: .*[\'\"].*(?<![!:=])={1}(?!=)(?![^\s:])
  • While looking at the f-string lexical analysis section, I noticed it is missing logic for the conversions (!s, !r, !a)

The second part is fairly easy to fix, the regex just needs to be updated.

The first part is what's giving me trouble. Based on the fact that all the f-string formatting code is commented out, I assume there has been some sort of mismatch in intent, since part of that is normalize_fstring_quotes. To me, this looks like a source of code duplication, since normalize_fstring_quotes will need to handle all these same edge cases. That leads to the easiest way to solve this, which is to disallow merges that change an f-string's quote. As for anything past that, I'm not sure what the best way is to prevent the code in merging and normalizing from desyncing.

[MeGaGiGaGon]

These are false positives that the regex that stops from quote flipping even though none are visible. "" f'{"= "}' "" f'{"=:"}'

[MeGaGiGaGon]

Another case I found that doesn't get prevented: "" f'{1:{""=}}' and "" f'{1:"{""=}"}'