Removing additional newlines left by block-like structures (solved)

[shakfu]

I'm using mako to generate cpp code. I found that if I use blocks, additional newlines are added as a result of the block tags. While this is not a huge deal, it wasn't a problem with the jinja2 output that mako is supposed to replace.

After doing some research on this, it seems there is no built-in switch where I could switch off these extra new lines and I would have to add a backslash to the end of the block tag. This works but it is manual and ugly.

There was a post on the mako google groups where it was suggested by the author of mako to use a preprocessor to solve this automatically.

I've tried to do this, but the problem is that a backslash is an escape character in regex and an escape character python but itself needs to be escaped so you have to do one of:

lookup = TemplateLookup(
    directories=[self.gx.sysdir + '/templates/cpp/'],
    preprocessor=lambda s: re.sub(r'(^<%block.+>)', '\\1\\\\', s, flags=re.MULTILINE)

This insert an unwanted \ in the output

or (note the r'..' literal)

lookup = TemplateLookup(
    directories=[self.gx.sysdir + '/templates/cpp/'],
    preprocessor=lambda s: re.sub(r'(^<%block.+>)', r'\1\\\\', s, flags=re.MULTILINE)

which inserts \\ in the output

So basically, there doesn't seem to be a way (at least to me) via regex where one can insert this backslash escape without inserting extraneous characters.

What about another method:

def escape_block_newlines(text):
    results = []
    for line in text.splitlines():
         if line.startswith('<%block'):
            line = line.strip() + '\\'
         results.append(line)
    return "\n".join(results)

The above doesn't work either.. This inserts an unwanted \ in the output as well.

I would appreciate some advice on how to resolve this small but irritating problem.

[shakfu]

Ok, another few iterations helped.

I finally figured out a preprocessor function which solves this issue more generally:

def escape_extra_newlines(text):
    patterns = ['<%block', '<%include'] # extend patterns as required
    results = []
    for line in text.splitlines():
        if any(line.startswith(p) for p in patterns):
            line += '\\'
        results.append(line)
    return '\n'.join(results)

Which can be used as follows:

lookup = TemplateLookup(
    directories=['templates'],
    preprocessor= escape_extra_newlines,
    default_filters=['str', 'trim'], # in case you also want to trim whitespace via .strip() 
)