openresty / replace-filter-nginx-module

Streaming regular expression replacement in response bodies
260 stars 68 forks source link

Slow large text processing #15

Open sunnybear opened 8 years ago

sunnybear commented 8 years ago

We have quite a huge RegExp (1-2k of symbols, 30-100 groups). And on large bodies (300-500kb) it slows nginx down (1-5 seconds x 2 GHz x 1 core) with huge memory consumption (1-3-10 Gb). Although current impact to the document is quite small (about 1k of replacements). Can provide body + RegExp, but it seems memory usage logic must be rewritten to fit complex cases.

agentzh commented 8 years ago

@sunnybear Will you provide a minimal and standalone test case so that we can run locally? You can send it to me privately if you prefer.

sunnybear commented 8 years ago

surely