search

hhatto / autopep8

A tool that automatically formats Python code to conform to the PEP 8 style guide.
https://pypi.org/project/autopep8/
MIT License
4.58k stars 291 forks source link

Taking too much memory and time when long line is found #318

Open davtoh opened 7 years ago

davtoh commented 7 years ago

hi, thank you for creating autopep8, it is a great tool to generate clean scripts and compliant to PEP8. I think i found room for improvement. i stumbled upon a limitation that could be related to bad memory usages or leaks. If in the code there is a long (indeed really long line e.g. a list of data) then the program takes a humongous space in memory (my 8Gb of ram could not handle them) and needless to say a lot of time too, it couldn't even finish the file after 30 minutes and i had to stop the program after a lot of effort to stabilize my pc.

Here is the compressed file, demonstrating this... In fact, the line could not fit in here. test_autopep8_long_line.zip

it generates ---> Applying global fix for W602 ---> Applying global fix for E265 ---> 2 issue(s) to fix {'W391': {8}, 'E501': {7}} ---> 1 issue(s) to fix {'E501': {7}}

Once again, much appreciated for this great tool, hope this can help improving it even more.

hhatto commented 7 years ago

pycodestyle execution time is long, now.

$ time pycodestyle -r test_autopep8_long_line.py
test_autopep8_long_line.py:7:80: E501 line too long (138397 > 79 characters)
test_autopep8_long_line.py:8:1: W391 blank line at end of file
pycodestyle -r test_autopep8_long_line.py  116.25s user 1.01s system 96% cpu 2:02.09 total
davtoh commented 7 years ago

Indeed the execution is long for pycodestyle. I tested again with the pypi and master versions.

Before, autopep8 Version: 1.3.1 (notice that i had to finish it): $ time autopep8 test_autopep8_long_line.py ^C real 31m59.445s user 28m17.664s sys 3m40.772s After, Version: 2.0a2: time autopep8 test_autopep8_long_line.py ^C real 36m3.652s user 31m49.256s sys 4m13.136s

I guess that either way the line is really very long, perhaps a warning could be issued... As soon as i have the time i will try at least debugging... I take it is using autopep8.fix_code() right?