christopher-ramirez / secretary

Take the power of Jinja2 templates to OpenOffice and LibreOffice.
Other
190 stars 48 forks source link

Slow rendering even with a tiny amount of data #44

Closed axerdan closed 6 years ago

axerdan commented 7 years ago

I've found out that if I try to render a simple document with a table cycling about a hundred records the rendering time is very slow.

Profiling the code revealed that almost all of the time is spent inside the function: "_encode_escape_chars"; I guess the regexp is poorly performing.

I tried monkey patching the class with:

Renderer._encode_escape_chars = lambda s, v: v

and the render was fast.

For comparison: normal render 7.5 sec, without that function: 0.3 sec

I know that this function correctly escapes new lines, tabs etc. but I think there should be a way to escape the values before begin rendered in the final xml.

If you need an example just let me know. Btw I'm using python 3.5.2 on Ubuntu

christopher-ramirez commented 7 years ago

Hello!

An example will be great!

Regards, Chris

axerdan commented 7 years ago

I've created an example that I think should be enough.

sec_test.zip

Comment the row 2 to test using the regex.

Time skipping the regex:

real    0m0.334s
user    0m0.324s
sys 0m0.008s

Time with regex:

real    0m3.705s
user    0m3.684s
sys 0m0.016s

Thanks in advance for your support!

christopher-ramirez commented 7 years ago

Hello! I'm introducing a new method in 76cb15e2 which it is performing much better than previous one. Would you please take a look into it.

Thank you for reporting this issue.

axerdan commented 6 years ago

Hello, I tested it and it works fine! Thanks for your support! Can you push it to pip? Thanks

christopher-ramirez commented 6 years ago

I've to still make more tests with this new change before committing them in a new version.

christopher-ramirez commented 6 years ago

I recently applied a patch (076067c35). This fixs a bug where the newly introduced code was escaping and encoding variable values inside a HREF attribute of link. This resulted in corrupted attributes values, e.g: A variable containing a \n char: href="{{ variable }}" => href="http://<text:line-break />domain.com"

christopher-ramirez commented 6 years ago

The new version is now published on Pip. The latest version is now 0.2.17.

Thank you for your support.