Closed CurroRodriguez closed 2 years ago
Updated in Monday
This was reported by a project that had a mixed of Python and JavaScript files. The JS part used node_modules
, which increased the number of files and folders exponentially. I haven't seen a problem with any Python projects, even large ones, so I will not spend time on this.
The task was implemented using a
FileFinder
class that searches for all matching files. This class extends lists for each folder in the tree, which I believe it is what is causing the performance issues. You can see that once all the files are compiled, deleting them it is not too bad even with the added logging. There are several alternatives to improve how the task works; unfortunately, they don't all work on every version of Python. There has been improvements to theglob
module, but they are only available in Python 3.5 and later. Another alternative might be using a combination ofos.walk
and thefnmatch
module. The complication to come up with a version that works on all versions of Python is that the task also supports a recursive option which if set to false (the default), it only searches the specified folder.