Instagram / LibCST

A concrete syntax tree parser and serializer library for Python that preserves many aspects of Python's abstract syntax tree
https://libcst.readthedocs.io/
Other
1.57k stars 192 forks source link

codemod unified-diff generates not applicable patch #788

Open kalaskow opened 2 years ago

kalaskow commented 2 years ago

Hello,

I've written a codemod which rewrites parameters in given function call. Details are not important. Also, I work in a project where we use Perforce as a source control. As a result I can't use default libcst.tool mode where files are just modified in place as I need to know upfront which files need modification. I've decided to use -u option which generates unified diff. I can preprocess such diff to extract files which require modification, open them and the apply the diff. Problem is the generated diff can't be applied when file is "short".

The problem

Generate diff:

$ python -m libcst.tool codemod transform_call.ConvertListsToListOfDicts commands/tests/integration_test/file.py -u=3 > /tmp/x.diff
Calculating full-repo metadata...
Executing codemod...
reformatted -

All done! ✨ 🍰 ✨
1 file reformatted.
Finished codemodding 1 files!
 - Transformed 1 files successfully.
 - Skipped 0 files.
 - Failed to codemod 0 files.
 - 0 warnings were generated.

Apply diff:

$ patch commands/tests/integration_test/file.py /tmp/x.diff
patching file commands/tests/integration_test/file.py
Hunk #1 FAILED at 3.
1 out of 1 hunk FAILED -- saving rejects to file commands/tests/integration_test/file.py.rej

I've compared what is the difference between diff generated by libcst.tool (x.diff) an real diff (y.diff):

$ python -m libcst.tool codemod transform_call.ConvertListsToListOfDicts commands/tests/integration_test/file.py
$ diff commands/tests/integration_test/file.py.backup commands/tests/integration_test/file.py -u > /tmp/y.diff
$ diff /tmp/x.diff /tmp/y.diff
1,3c1,3
< --- /home/bill/python_codemods/commands/tests/integration_test/file.py
< +++ /home/bill/python_codemods/commands/tests/integration_test/file.py
< @@ -5,8 +5,15 @@
---
> --- commands/tests/integration_test/file.py.backup    2022-09-27 22:59:06.473918900 +0200
> +++ commands/tests/integration_test/file.py   2022-09-28 01:06:42.863918900 +0200
> @@ -5,7 +5,14 @@
20d19
<

It seems that libcst.tool codemod -u adds one unneccessary line which prevents patch to apply such diff.

< @@ -5,8 +5,15 @@
---
> @@ -5,7 +5,14 @@

If I use -u=1 to limit context diff is generated correctly and can be applied without any problems. When diff happens to span over entire file (-u=3) then it cannot be applied.

Conclusion

Am I using wrong options or diff generation in some cases is wrong?

kalaskow commented 2 years ago

I am sorry I haven't posted any example to reproduce.

Please take this file to_reproduce.txt, rename it to to_reproduce.py and do the following:

python -m libcst.tool codemod add_trailing_commas.AddTrailingCommas to_reproduce.py -u > mod.patch
patch to_reproduce.py mod.patch

With LibCST version 0.4.7 I encounter the following error:

patching file to_reproduce.py
Hunk #1 FAILED at 1.
1 out of 1 hunk FAILED -- saving rejects to file to_reproduce.py.rej

Funnily enough, when I remove trailing blank line from to_reproduce.py I get:

patching file to_reproduce.py
Hunk #1 succeeded at 1 with fuzz 1.