search

dileepajayakody / semanticvectors

Automatically exported from code.google.com/p/semanticvectors
Other
1 stars 0 forks source link

LSA Heap Size #42

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago 
What steps will reproduce the problem?
1. I have indexed the whole english wikipedia using Lucene
2. I am trying to run the LSA program
 java -Xmx7g pitt.search.semanticvectors.LSA -dimension 200 - -minfrequency 10 /local/index 
3.

What is the expected output? What do you see instead?

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
        at ch.akuhn.edu.mit.tedlab.SMat.<init>(SMat.java:42)
        at pitt.search.semanticvectors.LSA.smatFromIndex(LSA.java:126)
        at pitt.search.semanticvectors.LSA.main(LSA.java:202)

What version of the product are you using? On what operating system?
semanticvectors-2.4.jar

Please provide any additional information below.

Please help me. I want to whether by any chance this can be fixed or should I 
look for tools like Mahout which has SVD. Thank you so much for your code, time 
and help

Original issue reported on code.google.com by bharathd...@gmail.com on 6 Jul 2011 at 5:31

GoogleCodeExporter commented 9 years ago 
As discussed in the thread at
http://groups.google.com/group/semanticvectors/browse_thread/thread/9ca1270f9017
b51d
your best options are to increase heap size and use swap space on disk, or look 
into alternatives such as Mahout.

Original comment by widd...@google.com on 6 Jul 2011 at 8:59