Can the quarkus-lucene-extension be deployed to some repository?

[dainiusjocas]

My little tool lmgrep uses your quarkus-lucene-extension as a dependency. Not having this library published in some maven repo is inconvenient when it comes to automate tests, builds, etc.

Can I deploy the library to a public maven registry like sonatype or Clojars?

[gunnarmorling]

Hey, wow, didn't expect this extension to be used by others at this point. Let me think about this for a bit, best would be to publish it via Quarkus Universe, so it shows up in the extension catalogue. I believe @sannegrinovero also mentioned that someone else may be working on a Lucene extension. I don't mind publishing mine, it's worth pointing out though that it has been created for a very specific use case: read-only indexes created at application build time. It may take a bit of effort to generalize it.

[dainiusjocas]

I was also surprised that your Lucene extension worked in my case :+1: it seems that the library handles also the case for creating and querying in-memory indices which is all I need at the moment.

Yes, the best thing would be to generalize the library and publish it in the Quarkus universe.

However, if you don't mind me publishing the library with an attribution etc. as is to smth like Clojars then I'll take care of the details and when a proper extension is ready I'd just remove it. If that's not too brutal it would serve my current needs.

[gsmet]

@gunnarmorling I think this should be discussed with @yrodiere to check if there's some interest in supporting Hibernate Search with Lucene in Quarkus. If so it would make more sense to have the extension in Quarkus core. I'm not exactly sure if your extension would fully enable Hibernate Search to work though.

[yrodiere]

Last time I checked this extension was only working when dealing with in-memory indexes? If so, I don't think this would be enough to address realistic use cases with Hibernate Search, for now.

At one point @gustavonalle was working with @gunnarmorling to make Lucene fully work in native mode, so that they could use it in a Quarkus-based Infinispan. If that went through, then I totally would be interested in having that in Core so that we could use it in both a Hibernate Search extension in Core, and Infinispan (which probably wouldn't be in Core). I don't know what's the status of that effort, though?

[gustavocoding]

The effort was put on hold because of the lack of Methodhandle support in GraalVM, but this is supported now apparently in https://github.com/oracle/graal/issues/2761 Even without that, @gsmet proposed a workaround for the MMapDirectory where it'd deactivate some code paths in Lucene that would make it work in most scenarios. So no blockers at the moment to have this as part of Quarkus.

I will dust off my branch and send a PR soon

[gustavocoding]

FYI, the release notes for 21.0 https://www.graalvm.org/release-notes/21_0/#2100

[dainiusjocas]

FYI, the release notes for 21.0 https://www.graalvm.org/release-notes/21_0/#2100

FYI, I've tried to build lmgrep without the quarkus extension, the compilation suceeded but failed at runtime.

https://github.com/dainiusjocas/lucene-grep/issues/27#issue-789818056

[gustavocoding]

I invite both of you to try the new Quarkus Lucene extension that has just been released:

<dependency>
    <groupId>io.quarkiverse.lucene</groupId>
    <artifactId>quarkus-lucene</artifactId>
    <version>0.1</version>
</dependency>

Please let me know if it works for your use cases!

[dainiusjocas]

I invite both of you to try the new Quarkus Lucene extension that has just been released:

<dependency>
    <groupId>io.quarkiverse.lucene</groupId>
    <artifactId>quarkus-lucene</artifactId>
    <version>0.1</version>
</dependency>

Please let me know if it works for your use cases!

Compilation succeeded but the executable failed at runtime with the following exception:

Exception in thread "main" java.lang.UnsupportedOperationException: Attribute class 'interface org.apache.lucene.analysis.tokenattributes.KeywordAttribute' not supported in the image
        at io.quarkiverse.lucene.runtime.AttributeCreator.create(AttributeCreator.java:45)
        at org.apache.lucene.util.AttributeFactory$StaticImplementationAttributeFactory.createAttributeInstance(AttributeFactory.java:15)
        at org.apache.lucene.util.AttributeSource.addAttribute(AttributeSource.java:213)
        at org.apache.lucene.analysis.snowball.SnowballFilter.<init>(SnowballFilter.java:57)
        at lmgrep.lucene.text_analysis$analyzer_constructor$fn__3606.invoke(text_analysis.clj:76)
        at lmgrep.lucene.text_analysis.proxy$org.apache.lucene.analysis.Analyzer$ff19274a.createComponents(Unknown Source)
        at org.apache.lucene.analysis.AnalyzerWrapper.createComponents(AnalyzerWrapper.java:136)
        at org.apache.lucene.analysis.Analyzer.tokenStream(Analyzer.java:199)
        at org.apache.lucene.document.Field.tokenStream(Field.java:513)
        at org.apache.lucene.index.memory.MemoryIndex.addField(MemoryIndex.java:380)
        at org.apache.lucene.monitor.DocumentBatch$SingletonDocumentBatch.<init>(DocumentBatch.java:112)
        at org.apache.lucene.monitor.DocumentBatch$SingletonDocumentBatch.<init>(DocumentBatch.java:105)
        at org.apache.lucene.monitor.DocumentBatch.of(DocumentBatch.java:60)
        at org.apache.lucene.monitor.Monitor.match(Monitor.java:256)
        at org.apache.lucene.monitor.Monitor.match(Monitor.java:276)
        at lmgrep.lucene$match_text.invokeStatic(lucene.clj:34)
        at lmgrep.lucene$match_monitor.invokeStatic(lucene.clj:77)
        at lmgrep.lucene$highlighter$fn__3557.invoke(lucene.clj:88)
        at lmgrep.grep$match_lines.invokeStatic(grep.clj:87)
        at lmgrep.grep$grep$fn__3498.invoke(grep.clj:113)
        at lmgrep.grep$grep.invokeStatic(grep.clj:113)
        at lmgrep.core$_main.invokeStatic(core.clj:21)
        at lmgrep.core$_main.doInvoke(core.clj:12)
        at clojure.lang.RestFn.applyTo(RestFn.java:137)
        at lmgrep.core.main(Unknown Source)

[dainiusjocas]

A little fix was needed for my project to work. PR here https://github.com/quarkiverse/quarkus-lucene/pull/4/files

@gustavonalle thanks for the nice work!

[gunnarmorling]

Excellent, thanks, Gustavo!