ConfigParser is very slow

[rayluo]

Hi @PierreQuentel , first of all, I personally want to thank you for your project(s) enrich my journey, not once, but twice. The first time was my learning web development at around 15 years ago, I started with your Karrigell and used it for a couple years. Fast forward to Dec 2020, this Brython project gave me some fun during the holidays. :-)

Today I'm writing to report an observation. The ConfigParser module in Brython is very slow.

foo = ConfigParser()
foo.read("foo.ini")
# value = foo.get("section", "key")

An INI file as small as 200+ bytes in 20+ lines would require 3~8 seconds to be read() (presumably also parse) into memory, even before I attempt to get(...) its content. CPython on same machine can finish it within several milli-seconds.

The real INI my project needs to read, happens to have ~500 KB and 40,000 lines. CPython can finish it within seconds, Brython simply hangs on read("big.ini") for 15+ minutes and then I simply aborted.

For now, my workaround is to avoid ConfigParser, and use json format instead. Reading a 1 MB json file in Brython can be finished within seconds. But I still wonder, whether the slow ConfigParser is a bug which could be fixed, or a current limitation (which we may better add a run-time warning message to tell developers to avoid it).

I'm running Brython in Firefox ESR.

---

By the way, this FAQ page suggests that:

Q : what is the performance of Brython compared to CPython ?

R : this page compares the execution time of a number of elementary operations between the latest version of Brython on Firefox and CPython. The results vary depending on operations, but overall the order of magnitude is the same.

While the order of magnitude is mostly the same, could it be these 2 operations are very frequently used in most Python program, thus causing Brython to be potentially much slower, if/when these snippets are wrapped inside some loop?

Test	Brython(100 = CPython)	Code
build dictionary	1198	for i in range(1000000): a = {0: 0}
function call, complex arguments	964	def f(x, y=0, *args, **kw): return x for i in range(100000): f(i, 5, 6, a=8)

[PierreQuentel]

Hi @rayluo. You can't imagine how happy I am to see former Karrigell users ! For a short period of time it was probably a decent option among the many Python web frameworks, but when Django and Flask arrived it was clear that they were much better. Fortunately, the landscape of Python implementations in the browser is much less crowdy...

I have tested ConfigParser with a reasonably big .ini file and yes, the Brython version is unbearably slow. I suspect that the re module might be the culprit: the version used in Brython is a pure-Python module, much slower than the C implementation in CPython, and the regular expression in configparser are too complicated to be easily converted to Javascript regexps. I am currently working on a Javascript version of the Python re engine, but it's not advanced enough to support configparser.

The tests for "build dictionary" and "function call with complex arguments" might also be involved. I have already worked a lot on them, and there is certainly room for improvement, but I don't think they can ever be the same as in CPython because of the differences between Python and Javascript.

I keep this issue high on my todo list, but I can't tell when it will be fixed...

[rayluo]

Hi @rayluo. You can't imagine how happy I am to see former Karrigell users !

@PierreQuentel , you can't imagine how (pleasantly) surprised I am, either, when I was learning Brython for days and then suddenly noticed a major contributor's name of this repo looked familiar... I was like "I must have read this name before... who was this guy... AHA!" :smile:

I keep this issue high on my todo list, but I can't tell when it will be fixed...

You do not have to. For the ConfigParser issue, your reasoning on re makes total sense. I think we do not have to fix it right now, because config files are not that popular nowadays anyway. Focus your effort on the more fundamental re, "build dictionary" and "function call with complex arguments" instead. But, meanwhile, I would suggest to add a temporary warnings.warn("configparser module is currently much slower than running on CPython. The current workaround is to switch your file to json format."). That's what I did in my current project. Even big json file can be read in almost constant time.

That honest warning would at least avoid developers spending time on debugging where the issue was. It would otherwise take a new Brython user quite a while to find out which line in my app was slow, when the entire Brython app already become unresponsive.

PS: I may report some other issues down the road. They will be in a separated issue.