search

simulatino / trimtrailingwhitespaces

Python script that detects textfiles and then removes trailing white spaces and more.
The Unlicense
11 stars 2 forks source link

Performance #24

Open tbeu opened 10 years ago

tbeu commented 10 years ago

Cleaning annotations takes long time: e.g. minutes on a single file package.mo or > one hour on modelica_buildings. Would be interesting which step actually takes such a long time?

ttws is a suitable candidate for parallelization on file level since files can be treated independently.

dietmarw commented 10 years ago

Yes I'm aware that whenever we enter the pyparsing part it really slows down a lot. I think there is a lot of potential for improvement. Like currently I'm basically triggering each pyparsing procedure twice (and this for each type of annotation) since I need to take special care just for the last occurrence of an annotation. There are bound to be better ways but my time does currently not allow to investigate this further. But it is a valid issue, I agree.

dietmarw commented 10 years ago

Btw, on my machine I get about 19 mins for cleaning up the Buildings library.

tbeu commented 10 years ago

Buildings+Districts? It was more than 60min on my machine on Win using Python 2.7.6 32bit.

dietmarw commented 10 years ago

Yes both and well I'm on Linux 64bit with ssd drive.