Out of memory& reading XML file issues

[GoogleCodeExporter]

What steps will reproduce the problem?
1.Download java file and xml files.
2.In java file update String file = 
"C:\\Sripati\\FileCompareson\\XmlFile\\file.xml"; for your xml file path 
updation.
3. Run java file post compilation.

What is the expected output? What do you see instead?
1. Out put should all xn:MeContext tag ID values. It returns nothing.

2. If I change Expression xpath1 = dog.addXPath("//*"); for all elements then 
it throws 
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.util.Arrays.copyOf(Unknown Source)
    at java.lang.AbstractStringBuilder.expandCapacity(Unknown Source)
    at java.lang.AbstractStringBuilder.append(Unknown Source)
    at java.lang.StringBuilder.append(Unknown Source)
    at java.lang.StringBuilder.append(Unknown Source)
    at java.util.AbstractCollection.toString(Unknown Source)
    at java.lang.String.valueOf(Unknown Source)
    at java.io.PrintStream.println(Unknown Source)
    at XMLDogSearching.main(XMLDogSearching.java:45)

What version of the product are you using? On what operating system?
Downloaded jlibs-r1627 from your download link. OS is Windows vista

Please provide any additional information below.
Please help me as we are planning to handle around 3GB XML file search but now 
it is through memory error for all elements display at 8MB file search.

Suggest me best way to do it. Or let me know your development API guide to use 
it in right way.

Original issue reported on code.google.com by sripatid...@gmail.com on 24 Aug 2012 at 12:47

Attachments:

• XmlFile.zip
• XMLDogSearching.java

[GoogleCodeExporter]

change following lines:
     Object result = results.getResult(xpath1);
     System.out.println(result);
to:
     results.printResult(System.out, xpath1);

Original comment by santhosh.tekuri@gmail.com on 26 Aug 2012 at 9:02

[GoogleCodeExporter]

[deleted comment]

[GoogleCodeExporter]

Thanks for your response. In mean while I used your latest july2012 released 
jars and //* all element xpath search returned result instead of OutofMemory 
message. 
Still struggling to extract node values that I requested in case 1(now same 
code started throughing org.jaxen.saxpath.SAXPathException: undeclared prefix: 
xn where was it already set down in line 32,33 for xn& es). Let me check with 
these and expecting similar support for XML tag value fetching current issue 
resolutions please.

Original comment by sripatid...@gmail.com on 26 Aug 2012 at 9:23

[GoogleCodeExporter]

By using this even jlibs-r1627 started working fine without any memory issue. 
But how can I get values to be displayed in console or into a collection 
instead of XPATH given tags only. Still I am not able to fetch xn:MeContext 
tags attribute id and xn:vsDataType& es:userLabel tag text values.

Original comment by sripatid...@gmail.com on 26 Aug 2012 at 9:55

[GoogleCodeExporter]

to fix undeclared prefix, you should do following:
            nsContext.declarePrefix("xn", "genericNrm.xsd");
            nsContext.declarePrefix("es", "EricssonSpecificAttributes.12.26.xsd");

Original comment by santhosh.tekuri@gmail.com on 26 Aug 2012 at 10:12

[GoogleCodeExporter]

to get value of attribute id: use following xpath:
    "//xn:SubNetwork/xn:SubNetwork/xn:MeContext/@id"
the result of above xpath will be Collection<NodeItem>.
iterate over this collection and print NodeItem.value;

do the same for other xpaths.

Original comment by santhosh.tekuri@gmail.com on 26 Aug 2012 at 10:22

[GoogleCodeExporter]

Thanks a lot...Let me check these and get back you shortly. Hope now it will 
work for me. 
Please let me know if you have any best alternate approach to fetch data from 
3GB XML file.

Original comment by sripatid...@gmail.com on 26 Aug 2012 at 10:30

[GoogleCodeExporter]

Also let me know if it will have any type lock on it. As I have to suggest it 
for client to use as free for their XML file analysis.

Original comment by sripatid...@gmail.com on 26 Aug 2012 at 10:32

[GoogleCodeExporter]

for 3GB xml file, better use instant results (this avoids buffering results in 
memory)

what does "type lock" mean???

Original comment by santhosh.tekuri@gmail.com on 26 Aug 2012 at 10:35

[GoogleCodeExporter]

Now it works for me. I have to explore for instant result as well. Thanks for 
your kind supports.

Is it license free as open source or not?

Original comment by sripatid...@gmail.com on 26 Aug 2012 at 10:42

[GoogleCodeExporter]

it is LGPL license

Original comment by santhosh.tekuri@gmail.com on 26 Aug 2012 at 10:43

[GoogleCodeExporter]

Thanks a lot for your prompt and kind supports. 

Now I can suggest client about it as a Lesser General Public License is used to 
license free software so that it can be incorporated into both free software 
and proprietary software.

Original comment by sripatid...@gmail.com on 26 Aug 2012 at 10:54

[GoogleCodeExporter]

Foe instant result I changed java file is attached, executed for same XML 
file(attached earlier), it threw ArrayIndexOutofBounds exception& its detail is 
attached in .log file. Please let me know steps which should be taken care by 
me.

Original comment by sripatid...@gmail.com on 26 Aug 2012 at 5:24

Attachments:

• XMLDogInstantSearching.java
• InstantSerchExc.log

[GoogleCodeExporter]

following lines should be done after adding all xpaths:
            Event event = dog.createEvent();
            event.setXMLBuilder(new DOMBuilder());
            event.setListener(...) 

Original comment by santhosh.tekuri@gmail.com on 27 Aug 2012 at 10:29

[GoogleCodeExporter]

now it worked but 
Object meContextResult = event.evaluate(meContextsXpaths);
Object mosResult = event.evaluate(vsMOXpath);
Object moParamValueResult = event.evaluate(moParamValueXpath);

These result object value returning null. How can I pick those node values into 
collections.

Original comment by sripatid...@gmail.com on 27 Aug 2012 at 10:43

[GoogleCodeExporter]

you should not use: event.evaluate(...)

the result is notified using: InstantEvaluationListener.onNodeHit(...)
the first argument is xpath expression and second argument is nodeitem.

Original comment by santhosh.tekuri@gmail.com on 27 Aug 2012 at 12:23

[GoogleCodeExporter]

I guess I have to write code and load into collections based on logic inside 
that overridden onNodeHit(). Please let me know if I am in wrong track.

Original comment by sripatid...@gmail.com on 28 Aug 2012 at 5:26

[GoogleCodeExporter]

yes. you are correct.

but, you lost the advantage of Instant evaluation.

Instant evaluation notifies your InstantListener when the node is hit, so that 
you process the node immediately and do not store it. if you store all nodes 
hit for an xpath in a collection, you might run into outofmemory.

if you wanted to collect them in collection, then you can simply use 
XPathResults as before(rather than using InstantEvaluation listener)

Original comment by santhosh.tekuri@gmail.com on 28 Aug 2012 at 5:42

[GoogleCodeExporter]

Ok thanks for your valuable suggestions...

Original comment by sripatid...@gmail.com on 28 Aug 2012 at 7:01

[GoogleCodeExporter]

Hey with the same instant result code, normal search and the same shared XML I 
have been using xpath "//xn:MeContext[@id='SFOe0400015'] 
//es:vsDataEUtranCellFDD/es:otdoaSuplActive/text()". 

It is returning three times where as its existence is once(search XML file for 
otdoaSuplActive in system searching technique and returning only one tag).

May I have any way to avoid this issue please?

Original comment by sripatid...@gmail.com on 29 Aug 2012 at 4:03

[GoogleCodeExporter]

I checked with xmllint command. and it gives three nodes. here is the output:

$ xmllint --shell /Users/santhosh/Downloads/XmlFile/file.xml 
/Users/santhosh/Downloads/XmlFile/file.xml:4: namespace warning : xmlns: URI 
configData.xsd is not absolute
    xmlns:gn="geranNrm.xsd" xmlns="configData.xsd">
                                                  ^
/ > setns xn=genericNrm.xsd es=EricssonSpecificAttributes.12.26.xsd
/ > cat 
//xn:MeContext[@id='SFOe0400015']//es:vsDataEUtranCellFDD/es:otdoaSuplActive/tex
t()
 -------
false
 -------
false
 -------
false
/ > exit

Original comment by santhosh.tekuri@gmail.com on 30 Aug 2012 at 4:56

[GoogleCodeExporter]

Please ignore...XML is having it three times.

Original comment by sripatid...@gmail.com on 30 Aug 2012 at 5:01

[GoogleCodeExporter]

Hi Santhosh,

I am using XMLDog api for xmlpath searching...but i am facing CPU issue while 
searching in 1.5 GB file. When I am searching in 1.5 GB file CPU utilization 
goes to 100% and system stop working...do you have any suggestion for same...?

Original comment by vijaygai...@gmail.com on 5 Sep 2012 at 10:28

[GoogleCodeExporter]

hi vijay,

If you provide thread dump, that might help to see what is happening.
may be some thread is going to infinite loop??

Original comment by santhosh.tekuri@gmail.com on 5 Sep 2012 at 11:31

[GoogleCodeExporter]

closing the issue

Original comment by santhosh.tekuri@gmail.com on 21 Sep 2012 at 8:05

• Changed state: Fixed

[GoogleCodeExporter]

Good co-ordinations... Thanks for your kind and prompt supports....

Original comment by sripatid...@gmail.com on 22 Sep 2012 at 4:13