double-quote-cython-strings producing uncythonizable f-strings

[mironleon]

System

Ubuntu 22.04

cython == 3.0.10 cython-lint == 0.16.2 python == 3.12.3

Minimum working example

file test.pyx

def function():
    bar = 10
    a = f"foo [{'bar'}]"
    print(a)

run

double-quote-cython-strings test.pyx

changes test.pyx to

def function():
    bar = 10
    a = f"foo [{"bar"}]"
    print(a)

running

cythonize -i test.pyx

leads to error

Error compiling Cython file:
------------------------------------------------------------
...
def function():
    a = f"foo [{"bar"}]"
                 ^
------------------------------------------------------------

test.pyx:2:17: empty expression not allowed in f-string

Error compiling Cython file:
------------------------------------------------------------
...
def function():
    a = f"foo [{"bar"}]"
                 ^
------------------------------------------------------------

test.pyx:2:17: missing '}' in format string expression

Error compiling Cython file:
------------------------------------------------------------
...
def function():
    a = f"foo [{"bar"}]"
                 ^
------------------------------------------------------------

Note that leaving the cython file untouched (using single quotes inside the f-string) cythonizes correctly. This might also be a cython issue

[MarcoGorelli]

thanks @mironleon for the report - looks like a bug here in cython-lint

[mironleon]

Thanks for the response. I have briefly stepped through the code, but I don't see an easy way to fix this. The code so far doesn't seem to have any context-awareness, just fixing one token at a time

[mironleon]

I had a longer look and found the FSTRING_START and FSTRING_END tokens in the tokenize library, which makes a rudimentary fix relatively simple (ignoring all quotes inside of f strings)

let me know what you think

[MarcoGorelli]

thanks! I'd originally vendored this code from https://github.com/pre-commit/pre-commit-hooks?tab=readme-ov-file#double-quote-string-fixer (and included their license in the LICENSES folder). they do the opposite (double to single). do they also have this issue?