Closed asfimport closed 17 years ago
asfimport
commented
17 years ago Michael McCandless (@mikemccand) (migrated from JIRA)
Just to add another datapoint: I've been using the wikipedia snapshot just before 05/27 (I think it was 04/22 or so but I can't access the download site right now to confirm) without any issues.
asfimport
commented
17 years ago Grant Ingersoll (@gsingers) (migrated from JIRA)
Here's a patch to just the build.xml that downloads from people.a.o
You still need all the other stuff (libs, patches) to make this work.
asfimport
commented
17 years ago Grant Ingersoll (@gsingers) (migrated from JIRA)
OK, I reran it and it went fine. Not sure what happened, but maybe just a hiccup on my machine.
I am going to commit.
asfimport
commented
17 years ago Grant Ingersoll (@gsingers) (migrated from JIRA)
Committed.
asfimport
commented
17 years ago Michael McCandless (@mikemccand) (migrated from JIRA)
I think somehow the wrong version (1.4.4) of Xerces was committed and named as lib/xerces-2.9.0.jar.
I'm hitting what I think is the same issue Steven is referring to above (https://issues.apache.org/jira/browse/LUCENE-848#action_1248988), this exception after \~350K docs:
Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: Internal Error: fPreviousChunk == NULL at org.apache.lucene.benchmark.utils.ExtractWikipedia.extract(ExtractWikipedia.java:184) at org.apache.lucene.benchmark.utils.ExtractWikipedia.main(ExtractWikipedia.java:199) Caused by: java.lang.RuntimeException: Internal Error: fPreviousChunk == NULL at org.apache.xerces.framework.XMLParser.parse(XMLParser.java:1111) at org.apache.lucene.benchmark.utils.ExtractWikipedia.extract(ExtractWikipedia.java:181) ... 1 more
I downloaded 1.4.4 of xerces.jar and "cmp" says it's the same file that's now checked in as lib/xerces-2.9.0.jar. When I download xerces 2.9.0 myself and overwrite this one in lib, the extraction finishes without errors.
asfimport
commented
17 years ago Grant Ingersoll (@gsingers) (migrated from JIRA)
OK, go ahead and commit it.
Grant Ingersoll Center for Natural Language Processing http://www.cnlp.org/tech/lucene.asp
Read the Lucene Java FAQ at http://wiki.apache.org/lucene-java/LuceneFAQ
asfimport
commented
17 years ago Grant Ingersoll (@gsingers) (migrated from JIRA)
This has been committed
Add support for using Wikipedia for benchmarking.
Migrated from LUCENE-848 by Steven Parkes, 1 vote, resolved Jul 08 2007 Attachments: LUCENE-848.txt (versions: 7), LUCENE-848-build.patch, WikipediaHarvester.java, xerces.jar (versions: 2), xml-apis.jar Linked issues:
2015