Extract other type of fields

[GoogleCodeExporter]

What steps will reproduce the problem?
1. Loading report with http://xdocreport.opensagres.cloudbees.net/loadReport 
(report attached)

What is the expected output? What do you see instead?
There is no fields in Data Model found.

What version of the product are you using? On what operating system?
- latest
- Win7

Please provide any additional information below.
Maybe because there is "MACROBUTTON" fields, not "MERGEFIELD"?
There is fields looks like { MACROBUTTON ##611010825271## Name }, and I need to 
extract this field and know it's name. FieldsExtractor used in loadReport and 
it didn't find any fields.

Original issue reported on code.google.com by MrSo...@gmail.com on 27 Dec 2012 at 11:15

Attachments:

• TempTest.docx

[GoogleCodeExporter]

I don't know MACROBUTTON and never tested wiht it.

But when I see your MACROBUTTON name it's not a Freemarker interpolation. It 
should be ${name} and not 611010825271

Original comment by angelo.z...@gmail.com on 27 Dec 2012 at 12:05

• Changed state: Accepted

[GoogleCodeExporter]

But I think you will have problem even with ${name} because MS Word split the 
name in several xml element. (It's the same problem with mergefiled and that's 
why processing is done by XDocReport).

So procesing for MACROBUTTON  should be done too.

Original comment by angelo.z...@gmail.com on 27 Dec 2012 at 12:09

[GoogleCodeExporter]

So, what am I gonna do now? Can I parse this reports with XDocReport or I have 
to pick other lib?

Original comment by MrSo...@gmail.com on 27 Dec 2012 at 12:24

[GoogleCodeExporter]

OK,

Today XDocReport doesn't support MACROBUTTON. But is it interesting?

1) if your goal is just to retrieve the name of MACROBUTTON, XDocReport is not 
done for that (her goal is to manage reporting). So I suggest you to do that 
with Apache POI XWPF.

2) if your goal is to use MACROBUTTON for report and if you explain me why you 
prefer using MACROBUTTON instead of MERGEFIELD for your report, and if it's 
very interesting, I could try to manage MACROBUTTON.

Original comment by angelo.z...@gmail.com on 27 Dec 2012 at 2:18

[GoogleCodeExporter]

Actually, I need to use MACROBUTTON for report. Why I prefer it? As I was told 
this templates was made by users, who are not programmers. They are using MS 
Word without any plug ins and it is not difficult. Is it interesting?

Original comment by MrSo...@gmail.com on 28 Dec 2012 at 7:47

[GoogleCodeExporter]

Hi,

I have seen quickly MACROBUTTON and it seems that it is used to manage 
checkbox, list of items etc. Those feature can be interested for XDocReport (eg 
: manage checked/unchecked with Java context, manage items coming from Java 
List context).

In your docx, you use MACROBUTTON, just for display simple text. So I tell me 
why using  MACROBUTTON and not MERGEFIELD? 

> As I was told this templates was made by users, who are not programmers.
But MERGEFIELD can be created too by user with MS Word. Using MERGEFIELD 
advantage are : 

 * standard mean to manage mail merge. 
 * XDocReport uses MERGEFIELD and it provides a XDocReport Macro which helps you to design your report (list of fields name are displayed in a a dialog). See http://code.google.com/p/xdocreport/wiki/DocxReportingQuickStart

> They are using MS Word without any plug ins and it is not difficult.
Using MERGEFIELD is the same thing. I have explained how to create at hand with 
MS Word MERGEFIELD at http://code.google.com/p/xdocreport/wiki/DocxDesignReport 
but I think it better to use the XDocReport Macro.

So using MACROBUTTON just for simple text, is not very interesting (MERGEFIELD 
exists for that). Using MACROBUTTON  for checkbox, listbox etc can be 
interested but I think it's hard task (preprocessing of MERGEFIELD is the 
hardest task that I have developped in XDocReport).

Regards Angelo

Original comment by angelo.z...@gmail.com on 28 Dec 2012 at 3:53

[GoogleCodeExporter]

It is only one simple template, but I have many of them (dozens and maybe 
more). And yes, there can be checkboxes and lists of items. Using MACROBUTTON 
is to have it benefits when it's needed and for display simple text too.
Is this really big difference preprocessing MACROBUTTON instead MERGEFIELD? 
(sorry for asking this, maybe I really don't understand) When you made it for 
MERGEFIELD it easier for you to switch to MACROBUTTON then for me to make it 
all from beginning =(
Well now I have to know, are you going to develop MACROBUTTON preprocessing?

Original comment by MrSo...@gmail.com on 28 Dec 2012 at 7:59

[GoogleCodeExporter]

One more question: how to specify encoding for output pdf?

Original comment by MrSo...@gmail.com on 28 Dec 2012 at 8:01

[GoogleCodeExporter]

>It is only one simple template, but I have many of them (dozens and maybe more)
Even if XDocReport supports MACROBUTTON, you will have to updateyour docx to 
use name with Freemarker syntax.

> Is this really big difference preprocessing MACROBUTTON instead MERGEFIELD?
I don't know, MACROBUTTON should be studied before.

> Well now I have to know, are you going to develop MACROBUTTON preprocessing?
I must study if it's possible to use MACROBUTTON, but I don't know when I will 
have time to do that.

> One more question: how to specify encoding for output pdf?
See Font encoding section at 
http://code.google.com/p/xdocreport/wiki/XWPFConverterPDFViaIText

Original comment by angelo.z...@gmail.com on 29 Dec 2012 at 12:15

[GoogleCodeExporter]

Maybe I don't even need Freemarker, all I need is to know Field name and know 
how to insert content in that Field.
About encoding: I'm using XDocReport in servlet. Can I specify encoding via GET 
parameters? Maybe like this /servlet?encoding=windows-1251&...

Original comment by MrSo...@gmail.com on 29 Dec 2012 at 7:01

[GoogleCodeExporter]

> Maybe I don't even need Freemarker, all I need is to know Field name and know 
how to insert content in that Field.

I think you have not understood how to XDocReport works. Freemarker/Velocity is 
used to replace value of the field. That's why you must use Freemarker/Velocity 
syntax for your field name.

With your docx you have 2 problems :

1) you use MACROBUTTON instead of MERGEFIELD
Do you have tested to set a MACROBUTTON name which follows Freemarker syntax 
like ${name}
I think it should be working, but as XDocReport don't process MACROBUTTON, the 
generated report will have again MACROBUTTON with the new value name. 

If you use MERGEFIELD, preprocesing is done and it remove the code of 
MERGEFIELD and the generated report contains only text (MERGEFIELD disappears).

2) you don't use Freemarker syntax (like ${name}) for your MACROBUTTON name.
I think you would like to use "name" instead of "${name}" and you could develop 
a preprocesor for that. (other idea that I have, it gives the capability to 
customize the field name syntax (see 
http://code.google.com/p/xdocreport/issues/detail?id=41) but nothing is 
developped).

But I suggest you to take time to read and follow 
http://code.google.com/p/xdocreport/wiki/DocxReportingQuickStart
And speak about this link to your user to explain her the advantage to use the 
XDocReport macro which was developped for user (and not for the programmer).

> Can I specify encoding via GET parameters? Maybe like this 
/servlet?encoding=windows-1251&...
No, but it's good idea, please create a new issue for that.

Original comment by angelo.z...@gmail.com on 29 Dec 2012 at 8:50

[GoogleCodeExporter]

1 and 2) Yes, I understood that Freemarker replace ${name} with "name_value", 
thats why I'll develop my own preprocessor which replace "##1231242345323## 
name" with value. But I still need tools to now what is in MACROBUTTON code 
(field name extractor) and how to replace that code (whole field) with value 
that I'll get with my preprocessor.

Original comment by MrSo...@gmail.com on 30 Dec 2012 at 7:03

[GoogleCodeExporter]

Created Issue 212

Original comment by MrSo...@gmail.com on 30 Dec 2012 at 7:03

[GoogleCodeExporter]

If you decide to develop your own preprocessor, I suggest you to replace 
"##1231242345323## name" with "##${name}## name" which uses Freemarker syntax. 
After field extractor will work. 

You must understand that preprocessor is done the first time. To develop a 
preprocessor you can use SAX or DOM. I suggest you to use DOM which is more 
easy and I suggest you to see 
http://code.google.com/p/xdocreport/source/browse/document/fr.opensagres.xdocrep
ort.document.docx/src/main/java/fr/opensagres/xdocreport/document/docx/preproces
sor/dom/DOMFontsPreprocessor.java

Once you have developped your preprocessor you can see 
http://code.google.com/p/xdocreport/source/browse/integrationtests/fr.opensagres
.xdocreport.core.test/src/test/java/fr/opensagres/xdocreport/document/docx/prepr
ocessor/fonts/DocxFontsWithFreemarkerTestCase.java to know how to use 
report#addPreprocessor

I will study MACROBUTTON to know if it's possible to manage it like MERGEFIELD 
(when I will find time).

Regards Angelo

Original comment by angelo.z...@gmail.com on 30 Dec 2012 at 10:12

[GoogleCodeExporter]

Hi,

I think I have understood your need. Your need is just replace name of 
MACROBUTON of your docx. The generated report will contains MACROBUTON but with 
name coming from Java context. Is that?

If it's that, I have attached a sample with preprocessor which manages that. 
But pay attention, perhaps it's bugged but it works with your attached docx.

See attached Java code. Hope I understand your need.

Regards Angelo

Original comment by angelo.z...@gmail.com on 2 Jan 2013 at 8:31

Attachments:

• MacroButtonNamePreprocessor.java
• TempTest.java

[GoogleCodeExporter]

Yes, it's look like I need. But how can I use it in servlet? What I have to 
@Override?

Original comment by MrSo...@gmail.com on 7 Jan 2013 at 12:33

[GoogleCodeExporter]

I have never done that but I will override the loadReport like this : 

-----------------------------------------------------------
protected IXDocReport loadReport( String reportId, XDocReportRegistry registry, 
HttpServletRequest request )
        throws IOException, XDocReportException
  IXDocReport report = susper.loadReport(reportId, registry, request);

  MacroButtonNamePreprocessor preprocessor = new MacroButtonNamePreprocessor();
  report.addPreprocessor( DocxConstants.WORD_DOCUMENT_XML_ENTRY, preprocessor );

  // IMPORTANT : call extractFields to force the execution of the preprocesor before
  // the report generation
  // 1.0.1 should fix that.
  report.extractFields( new FieldsExtractor() );

  // After the MacroButtonNamePreprocessor is executed, you have a list with MacroButtonName :
  // - MacroButtonName#getName() => Real name of the macro button
  // - MacroButtonName#getFreemarkerName() => Freemarker field name (ex : field0).
  List<MacroButtonName> names = preprocessor.getMacroButtonNames();
  // Register the list of macro button in the report to reuse it for the next report
  report.setData( MacroButtonNamePreprocessor.class.getName(), names );
  return report;
}
-----------------------------------------------------------

After you can use report.getData(MacroButtonNamePreprocessor.class.getName()) 
to retrieve the list of macro button name (ex: to display an HTML form which 
displays fields (like our demo, see 
http://code.google.com/p/xdocreport/source/browse/demo-webapp/src/main/java/fr/o
pensagres/xdocreport/webapp/LoadXDocReportServlet.java?repo=samples you can 
modify the doLoad code to use List<MacroButtonName> instead of MetaDataModel). 

Original comment by angelo.z...@gmail.com on 7 Jan 2013 at 1:53

[GoogleCodeExporter]

As I understood, you will override the loadReport and upload this on server, 
than I'll download new ".jar" library. After this I can override 
populateContext like it says in 
http://code.google.com/p/xdocreport/wiki/DocxReportingWEBApplicationServlet

-----------------
protected void populateContext(IContext context, String reportId,
                        HttpServletRequest request) throws IOException, XDocReportException {
    if (REPORT_ID.equals(reportId)) {
        IXDocReport report = getRegistryFromHTTPSession( request ).getReport( reportId );
        MacroButtonName extractor = report.getData( MacroButtonNamePreprocessor.class.getName() );
        List<MacroButtonName> fields = extractor.getFields();
        for ( MacroButtonName field : fields )
        {
            String name = field.getName();
            String value = "123"; // I'll get this string from DB
        context.put(name, value);
        }
    }
    // else manage another report
  }
----------------

Is this will replace all MACROBUTTON fields in report with this values?

Original comment by MrSo...@gmail.com on 7 Jan 2013 at 7:25

[GoogleCodeExporter]

Please takes time to read my code. I have commented my code to explain you 
MacroButtonName  code, so you must do : 

--------------------------------
String name = field.getFreemarkerName();
--------------------------------

and NOT: 

--------------------------------
String name = field.getName();
--------------------------------

Original comment by angelo.z...@gmail.com on 8 Jan 2013 at 12:06

[GoogleCodeExporter]

What about modified lib where "loadReport" was overridden? Have you done it? 
Where can I get it?

Original comment by MrSo...@gmail.com on 9 Jan 2013 at 11:26

[GoogleCodeExporter]

Sorry I don't understand your question?

Original comment by angelo.z...@gmail.com on 9 Jan 2013 at 11:30

[GoogleCodeExporter]

".jar"-file, where class AbstractProcessXDocReportServlet (see Comment 17).

Original comment by MrSo...@gmail.com on 9 Jan 2013 at 12:11

[GoogleCodeExporter]

Or I can override by myself add in "public class MyServlet extends 
AbstractProcessXDocReportServlet" this:

---------------------
@Override
protected IXDocReport loadReport( String reportId, XDocReportRegistry registry, 
HttpServletRequest request )
        throws IOException, XDocReportException
  IXDocReport report = super.loadReport(reportId, registry, request);

  MacroButtonNamePreprocessor preprocessor = new MacroButtonNamePreprocessor();
  report.addPreprocessor( DocxConstants.WORD_DOCUMENT_XML_ENTRY, preprocessor );

  // IMPORTANT : call extractFields to force the execution of the preprocesor before
  // the report generation
  // 1.0.1 should fix that.
  report.extractFields( new FieldsExtractor() );

  // After the MacroButtonNamePreprocessor is executed, you have a list with MacroButtonName :
  // - MacroButtonName#getName() => Real name of the macro button
  // - MacroButtonName#getFreemarkerName() => Freemarker field name (ex : field0).
  List<MacroButtonName> names = preprocessor.getMacroButtonNames();
  // Register the list of macro button in the report to reuse it for the next report
  report.setData( MacroButtonNamePreprocessor.class.getName(), names );
  return report;
}
-----------------------

Original comment by MrSo...@gmail.com on 9 Jan 2013 at 12:14

[GoogleCodeExporter]

I'm trying to add preprocessor when I populating context:

-------------------------------
    @Override
    protected void populateContext(IContext context, String reportId, HttpServletRequest request) throws IOException,
                                                                                                         XDocReportException {
        if (REPORT_ID2.equals(reportId)) {

            IXDocReport report = getRegistryFromHTTPSession(request).getReport(reportId);

            MacroButtonNamePreprocessor preprocessor = new MacroButtonNamePreprocessor();
            report.addPreprocessor(DocxConstants.WORD_DOCUMENT_XML_ENTRY, preprocessor);

            // IMPORTANT : call extractFields to force the execution of the preprocesor before
            // the report generation
            // 1.0.1 should fix that.
            report.extractFields(new FieldsExtractor());

            // After the MacroButtonNamePreprocessor is executed, you have a list with MacroButtonName :
            // - MacroButtonName#getName() => Real name of the macro button
            // - MacroButtonName#getFreemarkerName() => Freemarker field name (ex : field0).
            List<MacroButtonName> names = preprocessor.getMacroButtonNames();
            // Register the list of macro button in the report to reuse it for the next report
            report.setData(MacroButtonNamePreprocessor.class.getName(), names);

            for (MacroButtonName macroButtonName : names) {
                // Real name of the macro button
                System.err.println(macroButtonName.getName());
                // Freemarker field name (ex : field0).
                System.err.println(macroButtonName.getName());
                //context.put(name, value);
            }
        }
        // else manage another report
    }
---------------------------

and I get NullPointerException in report.addPreprocessor(...) Looks like 
getRegistryFromHTTPSession didn't get report.

Original comment by MrSo...@gmail.com on 9 Jan 2013 at 12:55

[GoogleCodeExporter]

What am I doing wrong when getting report with getRegistryFromHTTPSession?

Original comment by MrSo...@gmail.com on 9 Jan 2013 at 12:57

[GoogleCodeExporter]

> getRegistryFromHTTPSession didn't get report
But have you uploaded the report before? 

Do you have understood how to this servlet works? I suggest you to install our 
webapp demo and set debug breakpoint to understand how it works.

This servlet which extends UploadXDocReportServlet (see 
http://code.google.com/p/xdocreport/source/search?q=UploadXDocReportServlet&orig
q=UploadXDocReportServlet&btnG=Search+Trunk) is used in the load report page at 
http://xdocreport.opensagres.cloudbees.net/loadReport.jsp

When you upload a document, it calls UploadXDocReportServlet#doUpload 
so if you must override reportLoaded with this code:

-----------------------------------------------
@Override
protected void reportLoaded( IXDocReport report, HttpServletRequest request )
    {
            MacroButtonNamePreprocessor preprocessor = new MacroButtonNamePreprocessor();
            report.addPreprocessor(DocxConstants.WORD_DOCUMENT_XML_ENTRY, preprocessor);

            // IMPORTANT : call extractFields to force the execution of the preprocesor before
            // the report generation
            // 1.0.1 should fix that.
            report.extractFields(new FieldsExtractor());

            // After the MacroButtonNamePreprocessor is executed, you have a list with MacroButtonName :
            // - MacroButtonName#getName() => Real name of the macro button
            // - MacroButtonName#getFreemarkerName() => Freemarker field name (ex : field0).
            List<MacroButtonName> names = preprocessor.getMacroButtonNames();
            // Register the list of macro button in the report to reuse it for the next report
            report.setData(MacroButtonNamePreprocessor.class.getName(), names)
}
-----------------------------------------------

At this tep your document is uploaded and stores it in HTTP session registry.

When you select template engine, it calls doLoad to display fields. Here you 
can use 

List<MacroButtonName> names = 
report.getData(MacroButtonNamePreprocessor.class.getName())

to displays fields.

When button save is clicked it call doSave which register the report in the 
global registry.

At this step you have registered your report. After to generate report, you do 
: 

"public class MyServlet extends AbstractProcessXDocReportServlet" :

---------------------
 @Override
    protected void populateContext(IContext context, String reportId, HttpServletRequest request) throws IOException,
                                                                                                         XDocReportException {
        if (REPORT_ID2.equals(reportId)) {

            IXDocReport report = getRegistry(request).getReport(reportId);
            List<MacroButtonName> names = report.getData(MacroButtonNamePreprocessor.class.getName());

            for (MacroButtonName macroButtonName : names) {
                // Real name of the macro button
                System.err.println(macroButtonName.getName());
                // Freemarker field name (ex : field0).
                System.err.println(macroButtonName.getName());
                context.put(name, value);
            }
        }
        // else manage another report
    }
------------------------------

Please install demo webapp to understand how it works.

Original comment by angelo.z...@gmail.com on 9 Jan 2013 at 2:13

[GoogleCodeExporter]

I want to refuse to use Freemarker at all. I'm using "extends 
AbstractProcessXDocReportServlet" and get everything I need 
(List<MacroButtonName> names = preprocessor.getMacroButtonNames();). Can I 
merge MACROBUTTON fields with my content without using Freemarker?

--------------------------------------
    @Override
    protected IXDocReport loadReport(String reportId, XDocReportRegistry registry,
                                     HttpServletRequest request) throws IOException, XDocReportException {
        IXDocReport report = super.loadReport(reportId, registry, request);

        MacroButtonNamePreprocessor preprocessor = new MacroButtonNamePreprocessor();
        report.addPreprocessor(DocxConstants.WORD_DOCUMENT_XML_ENTRY, preprocessor);

        // IMPORTANT : call extractFields to force the execution of the preprocesor before
        // the report generation
        // 1.0.1 should fix that.
        report.extractFields(new FieldsExtractor());

        // After the MacroButtonNamePreprocessor is executed, you have a list with MacroButtonName :
        // - MacroButtonName#getName() => Real name of the macro button
        // - MacroButtonName#getFreemarkerName() => Freemarker field name (ex : field0).
        List<MacroButtonName> names = preprocessor.getMacroButtonNames();
        // Register the list of macro button in the report to reuse it for the next report
        report.setData(MacroButtonNamePreprocessor.class.getName(), names);

        IContext context = report.createContext();

        for (MacroButtonName macroButtonName : names) {
            // Real name of the macro button
            System.err.println(macroButtonName.getName());
            // Freemarker field name (ex : field0).
            System.err.println(macroButtonName.getFreemarkerName());
            // Id field
            System.err.println(macroButtonName.getId());

            context.put(macroButtonName.getFreemarkerName(), "content from ID");
        }

        return report;
    }
--------------------------------------

Can't understand why Freemarker write DEBUG:
DEBUG   13578   [freemark] (): Could not find template in cache, creating new 
one; id=["TempTest.docx!word/document.xml"["ru_RU",UTF-8,parsed] ]

Thats why when pupulating context it says:
freemarker.core.InvalidReferenceException: Expression field0 is undefined on 
line 2, column 612 in TempTest.docx!word/document.xml.

But I don't really need Freemarker at all. I don't want to replace NAME with 
{NAME} and trying to feed Freemarker with this modified report. How can I just 
replace <w:instrText xml:space="preserve">MACROBUTTON ##601033770294## 
NAME</w:instrText> with something, that tells docx that it is plain text (my 
content that I get from macroButtonName.getId() => id of content).

If it necessary to use freemarker, please tell me why it can't find modified 
report, and how to solve this problem.

Original comment by MrSo...@gmail.com on 10 Jan 2013 at 11:32

[GoogleCodeExporter]

ok, forget about post before =)))
I understood what you helped me to do. Now in result report I got "{MACROBUTTON 
##601033770296## newfield1}", but I want only "newfield1" - without any 
MACROBUTTON. How can I do this?

Original comment by MrSo...@gmail.com on 10 Jan 2013 at 11:57

[GoogleCodeExporter]

At first, if you wish to use XDocReport you need template engine. The 
MacroButtonNamePreprocessor that I have attached in this issue is a hack to not 
write freemarker field with Freemarker syntax and process macrobutton.

> Now in result report I got "{MACROBUTTON ##601033770296## newfield1}", but I 
want only "newfield1" - without any MACROBUTTON. How can I do this?

You must improve MacroButtonNamePreprocessor to do that, but it's not just 
replace "{MACROBUTTON ##601033770296## ${field0}}" with ${field0}, you must 
remove too the XML instrText. Welcome to the nightmare of preprocessing. More 
your docx is simply, but sometimes your MACROBUTOON can be splitted in several 
instrText (like mergefield).

That's why I suggested you to use MERGEFIELD. Since the start of XDOcReport, we 
have fixed a lot of problemes with preprocessing of MERGEFIELD.

I have no time to help you more. So you must study thee ooxml syntax to remove 
MACROBUTTON. 

I think XDocReport should manage MACROBUTTON, but today I don't see the benefit 
to manage MERGEFIELD and MACROBUTTON both. MERGEFIELD is the standard mean to 
manage mail merge. 

Perhaps there is benefit to use MACROBUTTON for listbox, checkbox? I don't 
know? If you show you the benfit to use MACROBUTTON, I could try to do that.

Good luck!

Regards Angelo

Original comment by angelo.z...@gmail.com on 10 Jan 2013 at 12:33

[GoogleCodeExporter]

Can I remove XML instrText in MacroButtonNamePreprocessor?

Original comment by MrSo...@gmail.com on 10 Jan 2013 at 12:57

[GoogleCodeExporter]

I'm trying to rename Node in MacroButtonNamePreprocessor:
document.renameNode(instrTextElt, instrTextElt.getNamespaceURI(), "w:t");
Why Node steel <w:instrText>?
When I'm trying to set attribute, Node changed: 
instrTextElt.setAttribute("xml:space", "preserve");

Original comment by MrSo...@gmail.com on 15 Jan 2013 at 11:34

[GoogleCodeExporter]

it's w3c DOM question. No time to investigate with your problem.
Try that:

-----------------------------------------------------------------------
document.renameNode(instrTextElt, instrTextElt.getNamespaceURI(), "t");
-----------------------------------------------------------------------

Original comment by angelo.z...@gmail.com on 15 Jan 2013 at 12:16