[Bug?] Infinite rewriting of newlines

[alvations]

Using this version:

% pip freeze | grep pyupgrade
pyupgrade==3.16.0

It seems like pyupgrade will infinitely add newlines on this file https://github.com/nltk/nltk/blob/aa72b961da4d8756f37a71fd858d5c95792626d7/nltk/test/unit/test_corpora.py

To replicate:

# Download  test_corpora.py
wget https://raw.githubusercontent.com/nltk/nltk/aa72b961da4d8756f37a71fd858d5c95792626d7/nltk/test/unit/test_corpora.py

# Download as original.py
wget https://raw.githubusercontent.com/nltk/nltk/aa72b961da4d8756f37a71fd858d5c95792626d7/nltk/test/unit/test_corpora.py -O original.py

pyupgrade test_corpora.py
pyupgrade test_corpora.py
pyupgrade test_corpora.py
pyupgrade test_corpora.py
pyupgrade test_corpora.py

And checking the output:

 % diff original.py test_corpora.py 
35a36,39
> 
> 
> 
> 
39a44,45
> 
> 

---

Background, this is affecting the NLTK repo at https://github.com/nltk/nltk/issues/3273 and CI/CD pyupgrade step is infinitely unsatisfy-able =) https://github.com/nltk/nltk/pull/3274

[asottile]

this is likely a regression in python 3.12.4 which will be fixed in the next patch release. can you try with 3.12.3?

[alvations]

Let me try that. Thanks again for the quick reply!

[asottile]

yeah this is the cpython 3.12.4 regression due to incorrect positions of triple-quoted-non-ascii-multiline strings - if you limit to 3.12.3 in your pre-commit workflow this will be avoided (or wait until 3.12.5 which is supposed to have a fix)

[purificant]

this is the cpython 3.12.4 regression due to incorrect positions of triple-quoted-non-ascii-multiline strings - if you limit to 3.12.3 in your pre-commit workflow this will be avoided (or wait until 3.12.5 which is supposed to have a fix)

Confirmed that in nltk ci pipeline we see the issue with 3.12.4, switching to 3.12.3 as a workaround works. Thank you. :+1: