Add docxRaw helper - Githubissues

pdecat commented 4 years ago

This is a tentative PR to add a docxRaw helper that allows to insert arbitrary XML anywhere in the docx document.

At first, I tried to simply use the {{{/}}} triple brackets to avoid escaping in the Handlebars templating engine, but it puts the XML in a w:t literal text tag and some docx editors does not support that well, e.g. Word Online displays a broken table and Libreoffice drops the run altogether.

Still WIP with regards to the following but submitted now to get early feedback if possible:

[x] decide on actual name of the helper: docxRaw
[x] add tests
[ ] add documentation

As an example, this allows to generate tables like this:

From the following template:

{{#docxTable rows=myrows columns=mycols}}{{docxRaw this}}{{/docxTable}}

And the following data:

{
    "myrows": [
        [
            "Virtual Machine",
            5000,
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>5100 ▲</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>5200 ▲</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"00FF00\"/></w:rPr><w:t>5100 ▼</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>5200 ▲</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>5300 ▲</w:t></w:r>"
        ],
        [
            "Database",
            2000,
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"00FF00\"/></w:rPr><w:t>1900 ▼</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>2000 ▲</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"00FF00\"/></w:rPr><w:t>1900 ▼</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>2000 ▲</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>2100 ▲</w:t></w:r>"
        ],
        [
            "Storage",
            500,
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>505 ▲</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>520 ▲</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"00FF00\"/></w:rPr><w:t>505 ▼</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>520 ▲</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>540 ▲</w:t></w:r>"
        ],
        [
            "Others",
            20,
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"CCCCCC\"/></w:rPr><w:t>20 ▶</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>30 ▲</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"00FF00\"/></w:rPr><w:t>20 ▼</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>30 ▲</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"CCCCCC\"/></w:rPr><w:t>30 ▶</w:t></w:r>"
        ],
        [
            "Total",
            8550,
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"00FF00\"/></w:rPr><w:t>8450 ▼</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>8980 ▲</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"00FF00\"/></w:rPr><w:t>8450 ▼</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>8980 ▲</w:t></w:r>",
            "<w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t>9300 ▲</w:t></w:r>"
        ]
    ],
    "mycols": [
        "Service",
        "2020/02",
        "2020/03",
        "2020/04",
        "2020/05",
        "2020/06",
        "2020/07"
    ]
}

bjrmatos commented 4 years ago

thank you for the PR and sorry that it took longer to review it. we were considering the docxRaw but so far it looks that "everything" can be done just with normal templating, even your example can be done this way.

tried to reproduce a bit what you are doing and this is the result so far, it seems it is possible to continue modifying to match your styles and content

Captura de pantalla 2020-11-04 a la(s) 4 20 41 p m

export-dynamic-table.zip

something to consider when working with the table is the table styles, especially for the bold text, sometimes the table styles can contain rules that go into conflict with the styles of the text. so the easy way to fix that is to disable bold text from the table styles rules. in my example i've already did that and everything works as expected, if the result of the condition is bold text it remains the same.

bjrmatos commented 4 years ago

do you have some other valid scenarios that justify introducing this helper? if yes, let us know to continue evaluating the need for a raw helper.

pdecat commented 4 years ago

Hi @bjrmatos, adding docxRaw is justified as ~opening that generated file in Libreoffice is broken:~

pdecat commented 4 years ago

Sorry, I spoke too fast before importing your sample.

Let me explain what we are doing with docxRaw: we are generating the whole tables from raw JSON and formatting the table content at the Javascript onBefore function level depending on field names and values as it provides much more flexibility.

For tables with dynamic column, we are even comparing values between cells to add colored arrows indicating the trend (up/same/down).

Here's an example of our current progress where the color is only applied to the arrow, but not to the amount:

The template is kept simple:

{{#docxTable rows=myrows columns=mycols}}{{docxRaw this}}{{/docxTable}}

The data provides the advanced formatting:

{"rows":[["Virtual Machine","€5,000.00","<w:r><w:t>€5,100.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>","<w:r><w:t>€5,200.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>","<w:r><w:t>€5,100.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"00FF00\"/></w:rPr><w:t> ▼</w:t></w:r>","<w:r><w:t>€5,200.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>","<w:r><w:t>€5,300.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>"],["Database","€2,000.00","<w:r><w:t>€1,900.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"00FF00\"/></w:rPr><w:t> ▼</w:t></w:r>","<w:r><w:t>€2,000.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>","<w:r><w:t>€1,900.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"00FF00\"/></w:rPr><w:t> ▼</w:t></w:r>","<w:r><w:t>€2,000.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>","<w:r><w:t>€2,100.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>"],["Storage","€500.00","<w:r><w:t>€505.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>","<w:r><w:t>€520.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>","<w:r><w:t>€505.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"00FF00\"/></w:rPr><w:t> ▼</w:t></w:r>","<w:r><w:t>€520.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>","<w:r><w:t>€540.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>"],["Others","€20.00","<w:r><w:t>€20.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"CCCCCC\"/></w:rPr><w:t> ▶</w:t></w:r>","<w:r><w:t>€30.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>","<w:r><w:t>€20.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"00FF00\"/></w:rPr><w:t> ▼</w:t></w:r>","<w:r><w:t>€30.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>","<w:r><w:t>€30.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"CCCCCC\"/></w:rPr><w:t> ▶</w:t></w:r>"],["Total","€8,550.00","<w:r><w:t>€8,450.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"00FF00\"/></w:rPr><w:t> ▼</w:t></w:r>","<w:r><w:t>€8,980.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>","<w:r><w:t>€8,450.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"00FF00\"/></w:rPr><w:t> ▼</w:t></w:r>","<w:r><w:t>€8,980.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>","<w:r><w:t>€9,300.00</w:t></w:r><w:r><w:rPr><w:rFonts w:ascii=\"FreeSans\" w:hAnsi=\"FreeSans\"/><w:color w:val=\"FF0000\"/></w:rPr><w:t> ▲</w:t></w:r>"]],"cols":["Service","2020/02","2020/03","2020/04","2020/05","2020/06","2020/07"]}

Note: the great advantage of formatting data in Javascript instead of the docx template is that I can have unit tests for all kind of things.

bjrmatos commented 4 years ago

we had some discussion internally and we decided to accept this PR, we think it makes sense to support this to support advanced users like you. however, we will like some changes, i'm going to specify the changes as reviews

pdecat commented 3 years ago

I've updated code for compatibility with nodejs 8.9 used in CI which does not support named capture groups:

     Error: Error while executing docx recipe. Invalid regular expression: /xml=(?<xml>[^{}\s]+)/: Invalid group

https://travis-ci.org/github/jsreport/jsreport-docx/builds/742646597#L341

See https://node.green/#ES2018-features--RegExp-named-capture-groups and https://github.com/nodejs/node/issues/19760

pdecat commented 3 years ago

The docxRaw helper no longer restricts what parent elements can be replaced.

bjrmatos commented 3 years ago

thank you for all your work here! we still have some ideas to make the logic simpler, but we don't see a reason to delay the merging. so we are merging the PR and plan to refactor the code a bit more by ourselves sometime later.

pdecat commented 3 years ago

Thanks @bjrmatos!

jsreport / jsreport-docx

Add docxRaw helper #23