datatonic / duke

Automatically exported from code.google.com/p/duke
0 stars 0 forks source link

Upgrade to Lucene 4.0 #84

Open GoogleCodeExporter opened 8 years ago

GoogleCodeExporter commented 8 years ago
Lucene 4.0 is out now, and we should try to use it.

Original issue reported on code.google.com by lar...@gmail.com on 31 Dec 2012 at 1:10

GoogleCodeExporter commented 8 years ago
Compiling causes about 10 compilation errors, and they're non-trivial to fix, 
because there's no documentation of how the API has changed. So basically, 
every single error requires figuring out what the changes are, and some them 
seem fairly deep.

Original comment by lar...@gmail.com on 14 Jan 2013 at 1:24

GoogleCodeExporter commented 8 years ago
Hi,

http://lucene.apache.org/core/4_0_0/MIGRATE.html

Won't this help ? 

Original comment by yann.bar...@gmail.com on 14 Jan 2013 at 1:33

GoogleCodeExporter commented 8 years ago
Yes, that helps a lot. Thank you! :)

Original comment by lar...@gmail.com on 14 Jan 2013 at 2:04

GoogleCodeExporter commented 8 years ago
While we waited, 4.1 came out, so let's upgrade to that instead.

Original comment by lar...@gmail.com on 25 Jan 2013 at 7:08

GoogleCodeExporter commented 8 years ago
[deleted comment]
GoogleCodeExporter commented 8 years ago
Yes, there are a number of new features I want to try, but haven't had time 
yet. Feel free to add issues for any specific ones not already in the issue 
list.

Also, if you want to commit patches for the 4.x migration (or anything else), 
please feel encouraged to do it.

Original comment by lar...@gmail.com on 25 Jan 2013 at 9:55

GoogleCodeExporter commented 8 years ago
Have started porting now, but tests fail. Will continue working on this 
tomorrow. 
http://code.google.com/p/duke/source/detail?r=cf75f271bed34b9b0e888684902551b93a
bdff0d&name=lucene4

Original comment by lar...@gmail.com on 1 Feb 2013 at 6:38

GoogleCodeExporter commented 8 years ago

Original comment by lar...@gmail.com on 1 Feb 2013 at 6:38

GoogleCodeExporter commented 8 years ago

Original comment by lar...@gmail.com on 1 Feb 2013 at 6:39

GoogleCodeExporter commented 8 years ago
Well, with the latest commit we have everything working on Lucene 4.1. However, 
it seems to run at roughly half the speed of Lucene 3.6. That's not very good. 
Need to consider what to do.

Original comment by lar...@gmail.com on 2 Feb 2013 at 9:19

GoogleCodeExporter commented 8 years ago
Sorry not to help more. At the moment, I'm focused on Elasticsearch tests...

Original comment by yann.bar...@gmail.com on 12 Feb 2013 at 8:46

GoogleCodeExporter commented 8 years ago
It turns out Lucene 4.0 is faster than both 4.1 and 3.6, so we'll go for 4.0 
for the moment, while we try to figure out how to improve performance on 4.1.

This question looks like it might contain a few clues: 
http://stackoverflow.com/questions/14851780/solve-terrible-performance-after-upg
rading-from-lucene-4-0-to-4-1

Original comment by lar...@gmail.com on 24 Feb 2013 at 11:44

GoogleCodeExporter commented 8 years ago
Tried adding this code:

import org.apache.lucene.codecs.FilterCodec;
import org.apache.lucene.codecs.StoredFieldsFormat;
import org.apache.lucene.codecs.lucene41.Lucene41Codec;
import org.apache.lucene.codecs.lucene40.Lucene40StoredFieldsFormat;

        cfg.setCodec(new CustomCodec());

  public final class CustomCodec extends FilterCodec {
    public CustomCodec() {
      super("CustomCodec", new Lucene41Codec());
    }

    public StoredFieldsFormat storedFieldsFormat() {
      return new Lucene40StoredFieldsFormat();
    }
  }  

It didn't work, though, because the CustomCodec needs to be registered in some 
kind of SPI framework so that Lucene can read the index. Will look at that 
later.

Original comment by lar...@gmail.com on 24 Feb 2013 at 3:58

GoogleCodeExporter commented 8 years ago

Original comment by lar...@gmail.com on 16 Mar 2013 at 11:40