When creating(didn't checked loading yet) a big nginx configuration crossplane is slow
Having a big nginx configuration with lots of files creating large ones can be very slow. Sometimes taking more than 30 seconds to create 4 files.
Cause
After some profiling I discovered that what takes the longest is deciding if a string requires quoting. While this is a nice feature, to automatically quote it can get in the way if you know your configuration doesn't require that and you want it to run fast.
Solution
The following patch adds a new parameter that toggles the quoting part if you know you don't need it. This makes the creation of the files almost 10x faster.
diff --git a/crossplane/builder.py b/crossplane/builder.py
index 049d224..8ffdc5e 100644
--- a/crossplane/builder.py
+++ b/crossplane/builder.py
@@ -50,59 +50,59 @@ def _needs_quotes(string):
return char in ('\\', '$') or expanding
def _replace_escape_sequences(match):
return match.group(1).decode('string-escape')
-def _enquote(arg):
- if not _needs_quotes(arg):
+def _enquote(arg, no_quote=False):
+ if no_quote or not _needs_quotes(arg):
return arg
if PY2:
arg = codecs.encode(arg, 'utf-8') if isinstance(arg, unicode) else arg
arg = codecs.decode(arg, 'raw-unicode-escape')
arg = repr(arg).replace('\\\\', '\\').lstrip('u')
arg = ESCAPE_SEQUENCES_RE.sub(_replace_escape_sequences, arg)
arg = unicode(arg, 'utf-8')
else:
arg = repr(arg).replace('\\\\', '\\')
return arg
-def build(payload, indent=4, tabs=False, header=False):
+def build(payload, indent=4, tabs=False, header=False, no_quote=False):
padding = '\t' if tabs else ' ' * indent
head = ''
if header:
head += '# This config was built from JSON using NGINX crossplane.\n'
head += '# If you encounter any bugs please report them here:\n'
head += '# https://github.com/nginxinc/crossplane/issues\n'
head += '\n'
def _build_block(output, block, depth, last_line):
margin = padding * depth
for stmt in block:
- directive = _enquote(stmt['directive'])
+ directive = _enquote(stmt['directive'], no_quote)
line = stmt.get('line', 0)
if directive == '#' and line == last_line:
output += ' #' + stmt['comment']
continue
elif directive == '#':
built = '#' + stmt['comment']
elif directive in EXTERNAL_BUILDERS:
external_builder = EXTERNAL_BUILDERS[directive]
built = external_builder(stmt, padding, indent, tabs)
else:
- args = [_enquote(arg) for arg in stmt['args']]
+ args = [_enquote(arg, no_quote) for arg in stmt['args']]
if directive == 'if':
built = 'if (' + ' '.join(args) + ')'
elif args:
built = directive + ' ' + ' '.join(args)
else:
built = directive
When creating(didn't checked loading yet) a big nginx configuration crossplane is slow Having a big nginx configuration with lots of files creating large ones can be very slow. Sometimes taking more than 30 seconds to create 4 files.
Cause After some profiling I discovered that what takes the longest is deciding if a string requires quoting. While this is a nice feature, to automatically quote it can get in the way if you know your configuration doesn't require that and you want it to run fast.
Solution The following patch adds a new parameter that toggles the quoting part if you know you don't need it. This makes the creation of the files almost 10x faster.