search

apache / lucene

Apache Lucene open-source search software
https://lucene.apache.org/
Apache License 2.0
2.67k stars 1.04k forks source link

Make Luke a Lucene/Solr Module [LUCENE-2562] #3636

Closed asfimport closed 5 years ago

asfimport commented 14 years ago

see "RE: Luke - in need of maintainer": http://markmail.org/message/m4gsto7giltvrpuf "Web-based Luke": http://markmail.org/message/4xwps7p7ifltme5q

I think it would be great if there was a version of Luke that always worked with trunk - and it would also be great if it was easier to match Luke jars with Lucene versions.

While I'd like to get GWT Luke into the mix as well, I think the easiest starting point is to straight port Luke to another UI toolkit before abstracting out DTO objects that both GWT Luke and Pivot Luke could share.

I've started slowly converting Luke's use of thinlet to Apache Pivot. I haven't/don't have a lot of time for this at the moment, but I've plugged away here and there over the past work or two. There is still a lot to do.

luke1.jpg

luke2.jpg

luke3.jpg

Luke-ALE-1.png

Luke-ALE-2.png

Luke-ALE-3.png

Luke-ALE-4.png

Luke-ALE-5.png

lukeALE-documents.png

Migrated from LUCENE-2562 by Mark Miller (@markrmiller), 11 votes, resolved Apr 24 2019 Attachments: LUCENE-2562.patch (versions: 3), LUCENE-2562-ivy.patch, LUCENE-2562-Ivy.patch (versions: 3), luke1.jpg, luke2.jpg, luke3.jpg, Luke-ALE-1.png, Luke-ALE-2.png, Luke-ALE-3.png, Luke-ALE-4.png, Luke-ALE-5.png, lukeALE-documents.png, luke-javafx1.png, luke-javafx2.png, luke-javafx3.png, screenshot-1.png, スクリーンショット 2018-11-05 9.19.47.png Linked issues:

Pull requests: https://github.com/apache/lucene-solr/pull/420, https://github.com/apache/lucene-solr/pull/490, https://github.com/apache/lucene-solr/pull/512

asfimport commented 9 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Hi Mark, Thank you for the response!

My patch needs ivy jar in lib/tools directory. (I've forgot to note that.) That does not introduce additional Java codes, but brings dependency resolution.

I've slowly started to catch up and play with (and modify) the codes... so I would like to create another patches to make progress.

asfimport commented 9 years ago

Upayavira (@upayavira) (migrated from JIRA)

Just a thought here - would it make sense to expose the luke functionality via the admin UI - use the Admin UI as the front end for luke and have it available to everyone right away?

Perhaps the reworked UI in AngularJS that I am working on might make this easier (see SOLR-5507).

asfimport commented 9 years ago

Torsten Krah (migrated from JIRA)

Just my 2 cents - i would like to see this as a Lucene module, not a SOLR one. Not everyone uses SOLR to use Lucene, so porting it to a SOLR only module would be no benefit in using lucene. Would be nice to still have a clean small solution to access lucene like luke does it now, not to be forced to use solr.

asfimport commented 9 years ago

Mark Miller (@markrmiller) (migrated from JIRA)

Agreed. This about making Luke a Lucene module, not about rewriting Luke as something else.

asfimport commented 9 years ago

Uwe Schindler (@uschindler) (migrated from JIRA)

My patch needs ivy jar in lib/tools directory. (I've forgot to note that.)

@Tomoko Uchida: We already use IVY in Lucene/Solr. Please use the framework that is already there through common-build.xml / module-build.xml. It is enough to place ivy.xml in the module directory and make the module include the module-build.xml file.

asfimport commented 9 years ago

Upayavira (@upayavira) (migrated from JIRA)

Absolutely, fair enough. Exposing it through Solr UI makes sense, restricting it does not.

asfimport commented 9 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Thanks everyone for all comments,

I recognize that the goal of this project is "making Luke a Lucene module". I think it will be nice if there is a REST API server for Admin UIs, and have some kind of pluggable architecture for external modules (like custom analyzers provided by Solr, Elsticsearch, etc.) But at first, I'd like to make it cover standalone Luke application features under Apache license.

Dear Uwe and all,

> We already use IVY in Lucene/Solr. Please use the framework that is already there through common-build.xml / module-build.xml. It is enough to place ivy.xml in the module directory and make the module include the module-build.xml file.

I wanted to use the Lucene's framework, but currently, this module is placed under http://svn.apache.org/repos/asf/lucene/sandbox/ , not http://svn.apache.org/repos/asf/lucene/dev/trunk

ivy.xml and ant tasks in my patch should be only temporary (for convenience in development,) until this module will be merged to trunk. So what can I do to smoothly integrate this module into Lucene trunk in the future. If there are better ways, please let me know...

Thanks

asfimport commented 9 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Patch updated. I've modified Overview tab only.

Progress and Status :

Pending tasks to be done:

I'm trying for decoders. It might need some sort of pluggable design (I believe Solr's decoders should be plugged, not built-in feature.) Suggestions / ideas welcome.

asfimport commented 9 years ago

ASF subversion and git services (migrated from JIRA)

Commit 1655103 from @markrmiller https://svn.apache.org/r1655103

LUCENE-2562: Add some Ivy support, support Lucene, Solr 4.10.3, Missing values in upper panel (index info) were all filled, Fields table are now sortable by field name and term counts. Tomoko Uchida.

asfimport commented 9 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Patch updated. I've modified Overview and Documents tab.

Progress and Status:

Pending tasks to be done:

I've added DecoderLoader interface and implemented SolrDecoderLoader class to make Luke work with or without Solr jars. I think we will be able to completely separate SolrDecoderLoader class from Luke's package later (or integrate this in Luke again if that is proper way.)

With this patch, if you want to use Luke with Solr's Decoder (wrapper for schema.FieldType,) you need to add a JVM option "luke.ext.decoder.loader" as below when start Luke.

$ java -cp "build" -Dluke.ext.decoder.loader=org.apache.lucene.luke.ext.SolrDecoderLoader org.apache.lucene.luke.ui.LukeApplication

My approach is not good, please let me know.

For SolrDecoder, need to modify ivy.xml as follows. (I could not include this modification in my patch, because 'svn diff' complains 'Cannot display: file marked as a binary type.') 

<!-- apache solr -->
    <dependency org="org.apache.solr" name="solr-core" rev="4.10.3"
                transitive="false"   // remove this line
                conf="solr->*,!sources,!javadoc"/>

I add some resources to 'LukeApplication_en.json', but I'm not a fluent English speaker, so please check and correct if there's any mistakes...

asfimport commented 9 years ago

Mark Miller (@markrmiller) (migrated from JIRA)

Awesome. I'll take a look soon.

asfimport commented 9 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Patch updated. Ive modified Documents tab. (A screen shot, "lukeALE-documents.png" is attached.)

Status and progress:

Pending tasks to be done (I'm going to implement them):

I understand this is not a rush job, many tasks still remain... I'd like to keep going and log updates here. Because components are well separated, I think anyone can join. :)

asfimport commented 9 years ago

Mark Miller (@markrmiller) (migrated from JIRA)

Great - been fairly busy lately, but I'll keep committing your work. Ping me with a reminder if I slack off too much.

asfimport commented 9 years ago

Mark Miller (@markrmiller) (migrated from JIRA)

I know it's homage to the original Luke default color, but we should really change the default background color :)

asfimport commented 9 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Progress and status:

Honestly there are no progress in development, I'd like to suggest about merge of this Jira issue and the Luke project now on Github. The (one of) Luke project's goal, "To port the thinlet UI to an ASL compliant license framework so that it can be contributed back to Apache Lucene.", is perfectly match the goal of this issue, I hope it would be desirable way. https://github.com/DmitryKey/luke/tree/master

After a short discussion at solr mailing list, Dmitry Kan, the maintainer of Luke, has created "pivot-luke" branch in luke repository. And I have merged all the latest pivot's version codes plugged in SVN to the branch.

The discussion is here: http://mail-archives.apache.org/mod_mbox/lucene-solr-user/201502.mbox/%3CCAHpHuj%3DZArPSbSLp2_d1X7OD_2V0NeBsDSRVizgTDTnBiAZPMQ%40mail.gmail.com%3E The branch for pivot's version is here: https://github.com/DmitryKey/luke/tree/pivot-luke

It now can be built by maven, you can check out and test it. We'll contribute back to Lucene it, I hope this is in not so distant future. Of course, developers are welcome.

> I know it's homage to the original Luke default color, but we should really change the default background color

I promise that, gray theme is also my favorite one! :)

I'll keep watching this ticket, any comments and suggestions are welcome. Thanks.

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

It's been a while... but I have ported Luke UI from Thinlet to JavaFX (I feel JavaFX is now better choice than other 3rd party Frameworks, what do you think?)

JavaFX version can be downloaded here: 

https://github.com/DmitryKey/luke/releases/tag/luke-7.3.1

The GitHub repository earns close to 900 stargagers! It's good time to contribute back to Lucene project but I am not sure what is the proper way. Let me know where should I start.

Anyway, please check it out and I want to hear your feedback!

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

I'll attach some screen shots to share what new UI looks like.

luke-javafx1.png

luke-javafx2.png

luke-javafx3.png

asfimport commented 6 years ago

Mark Miller (@markrmiller) (migrated from JIRA)

Awesome, that looks great!

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Thank you @markrmiller for response.

First I need some research for Lucene build system and try to fit the Github repository (built by maven) to it... any advise is welcome.

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Created a pull request on GitHub. Could you check it or is it better if I add a patch to the issue?

https://github.com/apache/lucene-solr/pull/420

asfimport commented 6 years ago

Steven Rowe (@sarowe) (migrated from JIRA)

@Tomoko Uchida: thanks for the pull request. I pulled it into a branch named jira/lucene-2562, along with a few fixes and additions, mostly to get ant precommit to succeed, which it does not yet do. I haven't tried to run the code yet :).

A few things I noticed:

  1. validate-source-patterns complains about non-static-final loggers: 
    validate-source-patterns:

    I fixed a few others like these, but ^^ have ctors that take a logger, apparently just to be able to turn logging off under testing. Is this really necessary? Right now I have a nocommit that allows ant precommit to ignore these, but it needs to be resolved.

  2. It should be possible to run Luke from a source checkout, but it's not now.
  3. I think Luke should be packaged with all other artifacts in Lucene's binary packages, but currently ant package-tgz and -zip don't include everything (e.g. scripts under bin/); instead, the build creates a .zip file that contains a copy of all the dependencies and the bin/ scripts. I think we can get rid of this Luke-only distribution, in favor of the Lucene distribution? I don't like the idea of including the Luke-with-all-dependencies .zip in the Lucene distribution, since it will include two copies of many modules' jars.
  4. forbidden-apis is very unhappy about classes in javafx.\* packages, e.g.: 
    [forbidden-apis] Scanning classes for violations...
[forbidden-apis] Forbidden class/interface use: javafx.beans.property.BooleanProperty [non-portable or internal runtime class]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.dto.search.SelectedField (SelectedField.java, field declaration of 'selected')
[forbidden-apis] Forbidden class/interface use: javafx.beans.property.SimpleBooleanProperty [non-portable or internal runtime class]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.dto.search.SelectedField (SelectedField.java:29)
[forbidden-apis] Forbidden class/interface use: javafx.beans.property.SimpleBooleanProperty [non-portable or internal runtime class]
[...]

    @uschindler: do you understand what ^^ is about?

asfimport commented 6 years ago

Uwe Schindler (@uschindler) (migrated from JIRA)

Hi, the forbiddenapis problems are caused by the way how javafx was exposed in previous versions of the JDK. With JDK8 it was shipped inside rt.jar, but on other versions it is not included at all and in Java 10 it's now part of module system. The portable runtime stuff only allows classes from the well known java packages, which does not yet include javafx.**: https://github.com/policeman-tools/forbidden-apis/blob/master/src/main/java/de/thetaphi/forbiddenapis/AsmUtils.java#L39-L40

The problem JavaFX is still not an official part of the Java SE platform, it's still optional! In addition this needs to be some special exceptional check, as javafx is only valid as runtime class in Java 8, in Java 7 it's not indluded.

If You run forbiddenapis with Java 10, it should pass. The reason is: Although it's shipped with the JDK and the class name is not in the above pattern, but because it's an external module - it works as expected (I have not verified this, maybe you can do it). The reason is the module name, it's external. With Java 8 we have no module names, so the check can only use the above whitelist of package names.

In short: That's a limitation of forbiddenapis and my ignorance for JavaFX. I'd suggest to redefine the forbiddenapis target in the luke module to not use the "jdk-non-portable" signature and instead the more static "jdk-internal" signature (maybe thorugh an ANT property). Nevertheless, as described above, JavaFX is a "non-portable" part of the JDK as it's not part of the spec. Not sure what the best way to solve this. I'd go for the workaround now and use "jdk-internal" instead of "jdk-non-portable" for the luke module (not for others, please).

asfimport commented 6 years ago

Uwe Schindler (@uschindler) (migrated from JIRA)

On top of that: Starting with Java 11, JavaFX is no longer shipped with the JDK, so forbiddenapis is correct: it's not portable.

See https://www.infoworld.com/article/3261066/java/javafx-will-be-removed-from-the-java-jdk.html

asfimport commented 6 years ago

Steven Rowe (@sarowe) (migrated from JIRA)

Thanks @uschindler, I put in an Ant property to control it.

That reveals a lot forbidden-api violations left to tackle now:

-check-forbidden-all:
[forbidden-apis] Reading bundled API signatures: jdk-unsafe-1.8
[forbidden-apis] Reading bundled API signatures: jdk-deprecated-1.8
[forbidden-apis] Reading bundled API signatures: jdk-internal-1.8
[forbidden-apis] Reading bundled API signatures: jdk-reflection
[forbidden-apis] Reading API signatures: /Users/sarowe/git/lucene-solr/lucene/tools/forbiddenApis/base.txt
[forbidden-apis] Reading API signatures: /Users/sarowe/git/lucene-solr/lucene/tools/forbiddenApis/lucene.txt
[forbidden-apis] Loading classes to check...
[forbidden-apis] Scanning classes for violations...
[forbidden-apis] Forbidden method invocation: java.util.Locale#toString() [use Locale#toLanguageTag() for a standardized BCP47 locale name]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.fragments.search.QueryParserController (QueryParserController.java:159)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.overview.OverviewImpl (OverviewImpl.java:159)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.overview.OverviewImpl (OverviewImpl.java:164)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes() [Uses default charset]
[forbidden-apis]   in org.apache.lucene.luke.models.analysis.TestAnalysisImpl (TestAnalysisImpl.java:93)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.dto.overview.TermCount (TermCount.java:29)
[forbidden-apis] Forbidden method invocation: java.io.PrintStream#<init>(java.io.OutputStream,boolean) [Uses default charset]
[forbidden-apis]   in org.apache.lucene.luke.app.util.TextAreaPrintStream (TextAreaPrintStream.java:38)
[forbidden-apis] Forbidden method invocation: java.lang.String#getBytes() [Uses default charset]
[forbidden-apis]   in org.apache.lucene.luke.app.util.TextAreaPrintStream (TextAreaPrintStream.java:49)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.QueryParserConfig (QueryParserConfig.java:235)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.QueryParserConfig (QueryParserConfig.java:236)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.QueryParserConfig (QueryParserConfig.java:237)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.QueryParserConfig (QueryParserConfig.java:238)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.QueryParserConfig (QueryParserConfig.java:239)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.QueryParserConfig (QueryParserConfig.java:240)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.QueryParserConfig (QueryParserConfig.java:241)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.QueryParserConfig (QueryParserConfig.java:242)
[forbidden-apis] Forbidden method invocation: java.util.Locale#toString() [use Locale#toLanguageTag() for a standardized BCP47 locale name]
[forbidden-apis]   in org.apache.lucene.luke.models.search.QueryParserConfig (QueryParserConfig.java:243)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.QueryParserConfig (QueryParserConfig.java:243)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.QueryParserConfig (QueryParserConfig.java:244)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.QueryParserConfig (QueryParserConfig.java:245)
[forbidden-apis] Forbidden method invocation: java.lang.String#toLowerCase() [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.LukeController$ColorTheme (LukeController.java:302)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.LukeController$ColorTheme (LukeController.java:302)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.OverviewController (OverviewController.java:210)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.OverviewController (OverviewController.java:212)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.DocumentsController (DocumentsController.java:225)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.DocumentsController (DocumentsController.java:302)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.DocumentsController (DocumentsController.java:325)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.DocumentsController (DocumentsController.java:349)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.commits.CommitsImpl (CommitsImpl.java:110)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.commits.CommitsImpl (CommitsImpl.java:123)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.commits.CommitsImpl (CommitsImpl.java:134)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.commits.CommitsImpl (CommitsImpl.java:151)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.commits.CommitsImpl (CommitsImpl.java:169)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.commits.CommitsImpl (CommitsImpl.java:187)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.commits.CommitsImpl (CommitsImpl.java:204)
[forbidden-apis] Forbidden method invocation: java.util.concurrent.Executors#newSingleThreadExecutor() [Spawns threads with vague names; use a custom thread factory (Lucene's NamedThreadFactory, Solr's DefaultSolrThreadFactory) and name threads so that you can tell (by its name) which executor it is associated with]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.dialog.menubar.CheckIndexController (CheckIndexController.java:120)
[forbidden-apis] Forbidden method invocation: java.util.concurrent.Executors#newSingleThreadExecutor() [Spawns threads with vague names; use a custom thread factory (Lucene's NamedThreadFactory, Solr's DefaultSolrThreadFactory) and name threads so that you can tell (by its name) which executor it is associated with]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.dialog.menubar.CheckIndexController (CheckIndexController.java:184)
[forbidden-apis] Forbidden method invocation: java.text.MessageFormat#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.util.MessageUtils (MessageUtils.java:41)
[forbidden-apis] Forbidden method invocation: java.lang.String#<init>(byte[]) [Uses default charset]
[forbidden-apis]   in org.apache.lucene.luke.app.util.TextAreaAppender (TextAreaAppender.java:57)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.dto.documents.TermPosting (TermPosting.java:32)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.documents.DocumentsImpl (DocumentsImpl.java:111)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.documents.DocumentsImpl (DocumentsImpl.java:149)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.documents.DocumentsImpl (DocumentsImpl.java:176)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.documents.DocumentsImpl (DocumentsImpl.java:205)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.documents.DocumentsImpl (DocumentsImpl.java:233)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.documents.DocumentsImpl (DocumentsImpl.java:256)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.documents.DocumentsImpl (DocumentsImpl.java:284)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.documents.DocumentsImpl (DocumentsImpl.java:302)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.documents.DocumentsImpl (DocumentsImpl.java:311)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.documents.DocumentsImpl (DocumentsImpl.java:320)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.models.commits.File (File.java:30)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.models.commits.File (File.java:31)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.dialog.search.ExplanationController (ExplanationController.java:74)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.AnalysisController (AnalysisController.java:159)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.AnalysisController (AnalysisController.java:164)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.AnalysisController (AnalysisController.java:165)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.AnalysisController (AnalysisController.java:165)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.AnalysisController (AnalysisController.java:172)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.AnalysisController (AnalysisController.java:178)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.AnalysisController (AnalysisController.java:178)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.AnalysisController (AnalysisController.java:190)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.dto.documents.TermVector (TermVector.java:41)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.SearchImpl (SearchImpl.java:150)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.SearchImpl (SearchImpl.java:185)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.SearchImpl (SearchImpl.java:224)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.SearchImpl (SearchImpl.java:236)
[forbidden-apis] Forbidden method invocation: java.util.Arrays#copyOfRange(java.lang.Object[],int,int) [Prefer using ArrayUtil as Arrays#copyOfRange fills zeros for bad bounds]
[forbidden-apis]   in org.apache.lucene.luke.models.search.SearchImpl (SearchImpl.java:326)
[forbidden-apis] Forbidden method invocation: java.util.Arrays#copyOfRange(java.lang.Object[],int,int) [Prefer using ArrayUtil as Arrays#copyOfRange fills zeros for bad bounds]
[forbidden-apis]   in org.apache.lucene.luke.models.search.SearchImpl (SearchImpl.java:356)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.SearchImpl (SearchImpl.java:424)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.SearchImpl (SearchImpl.java:452)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.analysis.AnalysisImpl (AnalysisImpl.java:85)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.analysis.AnalysisImpl (AnalysisImpl.java:189)
[forbidden-apis] Forbidden method invocation: java.util.concurrent.Executors#newSingleThreadExecutor() [Spawns threads with vague names; use a custom thread factory (Lucene's NamedThreadFactory, Solr's DefaultSolrThreadFactory) and name threads so that you can tell (by its name) which executor it is associated with]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.dialog.menubar.OptimizeController (OptimizeController.java:119)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.dialog.menubar.OpenIndexController (OpenIndexController.java:155)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.dialog.menubar.OpenIndexController (OpenIndexController.java:160)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.dialog.menubar.OpenIndexController (OpenIndexController.java:161)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.desktop.PreferencesImpl (PreferencesImpl.java:51)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.desktop.PreferencesImpl (PreferencesImpl.java, method declaration of 'iniFile()')
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.desktop.PreferencesImpl (PreferencesImpl.java:140)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.desktop.PreferencesImpl (PreferencesImpl.java:34)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.SearchController (SearchController.java:291)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.SearchController (SearchController.java:435)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.SimilarityConfig (SimilarityConfig.java:93)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.SimilarityConfig (SimilarityConfig.java:94)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.SimilarityConfig (SimilarityConfig.java:95)
[forbidden-apis] Forbidden method invocation: java.lang.String#format(java.lang.String,java.lang.Object[]) [Uses default locale]
[forbidden-apis]   in org.apache.lucene.luke.models.search.SimilarityConfig (SimilarityConfig.java:96)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.LogsController (LogsController.java:29)
[forbidden-apis] Forbidden class/interface use: java.io.File [Use NIO.2 instead]
[forbidden-apis]   in org.apache.lucene.luke.app.controllers.LogsController (LogsController.java:29)
[forbidden-apis] Scanned 161 class file(s) for forbidden API invocations (in 0.44s), 87 error(s).
asfimport commented 6 years ago

Uwe Schindler (@uschindler) (migrated from JIRA)

Thanks, looks good in your commit.

I am still unhappy that Luke will stop working with Java 11 (e.g., which will be the default after September's Update of Ubuntu 18.04).

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Thank you for the comments and detailed explanation.

> On top of that: Starting with Java 11, JavaFX is no longer shipped with the JDK, so forbiddenapis is correct: it's not portable.

I have not caught up with it. I cannot find out better solutions yet, but I'll try to look for alternative UI framework that can go with future JDK updates. Replacing UIs should be possible because core luke functionalities and UI components are well separated (I hope that,) though some extra work is needed.

If you have any suggestions, please let me know. :)

asfimport commented 6 years ago

Robert Muir (@rmuir) (migrated from JIRA)

I also found this issue related to the java 11 changes:

https://github.com/javafxports/openjdk-jfx/issues/52

Of course there are licensing aspects to think of too?

asfimport commented 6 years ago

Uwe Schindler (@uschindler) (migrated from JIRA)

We cannot link to GPL so this is a bummer. I don't think JavaFX, unless shipped with the JDK is a viable option for Apache projects.

I think we should ask ASF legal how to proceed?

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

As you know, in the ASF legal page, GNU GPL including GNU Classpath exceptions is listed in the Category X list (honestly, I read this terms yesterday and not prepared to handle such legal matters.) https://www.apache.org/legal/resolved.html#category-x

To not to complicate matters I'd like to withdraw the merge request for now. (Thanks for all your comments so far!) While contributing back Luke to Apache Lucene project is the our long-term goal, we can continually deliver Luke via current GitHub project. For keeping Luke source codes compatible with ALv2, Self-contained application package is the one of feasible options for users they do not (or cannot) install JavaFX, in my first impression. https://docs.oracle.com/javase/8/docs/technotes/guides/deploy/self-contained-packaging.html

Of course discussions and suggestions are appreciated!

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Let me add one more thing, I'm afraid that I cannot ask to and negotiate with ASF on my own about complicated matters with my limited knowledge about licensing and English vocabulary.

But if you think there are chances to proceed, please guide me :)

asfimport commented 6 years ago

Steven Rowe (@sarowe) (migrated from JIRA)

We cannot link to GPL so this is a bummer. I don't think JavaFX, unless shipped with the JDK is a viable option for Apache projects. I think we should ask ASF legal how to proceed?

As you know, in the ASF legal page, GNU GPL including GNU Classpath exceptions is listed in the Category X list (honestly, I read this terms yesterday and not prepared to handle such legal matters.) https://www.apache.org/legal/resolved.html#category-x

I'm afraid that I cannot ask to and negotiate with ASF on my own about complicated matters with my limited knowledge about licensing and English vocabulary.

I will make an INFRA JIRA and ask if an exception can be made for OpenJFX as a dependency (vs. bundled with the JRE, which is allowed).

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

@sarowe Thank you! Hope there will be good news.

asfimport commented 6 years ago

Steven Rowe (@sarowe) (migrated from JIRA)

I created https://issues.apache.org/jira/browse/LEGAL-396 to ask for an exception to allow Lucene to depend on OpenJFX.

asfimport commented 6 years ago

Dmitry Kan (migrated from JIRA)

Hi @sarowe thanks for your support with filing the ticket. Looking to solve this one way or another.

Thanks @Tomoko Uchida for your contribution and research so far!

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Hi,

I found a similar issue in ASF LEGAL.

"Should GPL+CPE move to Cat B?" https://issues.apache.org/jira/projects/LEGAL/issues/LEGAL-336

Seems the answer is definitely NO.

When sticking to JavaFX we probably cannot donate Luke to Apache products, though I like sophisticated JavaFX API and beautiful components. I'm now planning to port the UI layer from JavaFX to Swing.

Luke is getting more and more popular in GitHub users (dear committers, how many of you use Luke now?) I'm not sure why a desktop application gains popularity in the web browser era. But if many users need Luke, I think it should become an official member of Lucene in any form...

asfimport commented 6 years ago

Michael Sokolov (@msokolov) (migrated from JIRA)

Random, out of left field question here, please feel free to ignore, but have you considered porting to an HTTP/HTML gui? It could still be a standalone app using a little embedded HTTP service like this one. I for one have wanted to run Luke on a remote "cloud" server, but using X Windows to bring up the GUI locally is a pretty big barrier (especially if your local machine is not UNIX!).

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

@msokolov Thanks comments, sure, a tiny http server is always on my mind. (I am a just another web engineer, I know there are plenty of nice & light webapp frameworks...)

However,  I'm not sure that adding web assets (html, css, js) to Lucene, pure Java library, is acceptable. May be server side rendering (JavaScript free) is ok? I have no idea.

But web GUI perhaps get the point, when we are in a little bit dark period for client applications now.

Apache Pivot, first Mark Miller introduced here, has not been updated for one year. So I've switched to JavaFX as a promising framework, this is decoupled from JDK and blocked for Apache projects (at least for now.)

 

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Sorry for unfocused comment, I have no idea about what's the best.

So Swing based UI (plus web one, if someone can work for this) could be the starting point because it is not likely to obsoleted or removed from JDK in a while :)

asfimport commented 6 years ago

Alexandre Rafalovitch (@arafalov) (migrated from JIRA)

Couple of points to consider:

asfimport commented 6 years ago

Torsten Krah (migrated from JIRA)

Please don't make it a part of solr. The benefit of Luke is to have it standalone so you can use it just with plain Lucene. I am not using solr, but Lucene, so I would like to see it only with Lucene distribution and not be dependent on solr. My 2 cents about this.

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Sure, this is LUCENE's issue  ;)

asfimport commented 6 years ago

Dmitry Kan (migrated from JIRA)

@arafalov thanks for your input! Can you please elaborate on 'If Luke is supposed to be part of Lucene-only distribution, I guess the discussion is a bit more complicated' ?

asfimport commented 6 years ago

Alexandre Rafalovitch (@arafalov) (migrated from JIRA)

I just meant that Lucene does not have a server+HTML+CSS, so any UI addition will be a non-trivial discussion. I did not have any deeper insight than that.

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Well..., I am now working for completing a java desktop Luke, in any case.

If someone is strongly interested in a http server & web app version, feel free to fork the github repository ( https://github.com/DmitryKey/luke ) and develop it. I am the one long for a web version Luke that can access indexes in remote server, and will glad to cooperate.

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

@msokolov hi, I found a web based Luke project, with a rest api server and modern js technologies. This might be what you are wanting? 

http://www.flax.co.uk/blog/2017/02/24/release-1-0-marple-lucene-index-detective/

I've not tried this yet, for your information.

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Hi,

as we announced in the Lucene/Solr mailing lists, Luke was re-implemented on top of Swing. https://github.com/DmitryKey/luke

The code is licensed under ALv2 and Swing is the part of JDK, so I think there is no obstacle to make it a Lucene submodule.

I would like to create another patch and restart this issue, after just fixing styles and colors. The draft patch will be ready for review in the next few weeks or so but I am not sure about when I should cut the feature branch for it. (Seems like Lucene 8.0 release workflow will be kicked off soon.) Don't I have to care about the major version release procedure? Can you please give me some advice?

Thanks.

asfimport commented 6 years ago

Steven Rowe (@sarowe) (migrated from JIRA)

bq. as we announced in the Lucene/Solr mailing lists, Luke was re-implemented on top of Swing.

Wow, that was fast, congratulations!

The code is licensed under ALv2 and Swing is the part of JDK, so I think there is no obstacle to make it a Lucene submodule.

+1

The draft patch will be ready for review in the next few weeks or so but I am not sure about when I should cut the feature branch for it. (Seems like Lucene 8.0 release workflow will be kicked off soon.) Don't I have to care about the major version release procedure?

Once you put up a patch here I can commit it to the branch I already cut for this issue: jira/lucene-2562 (mentioned in a previous comment of mine on this issue). Depending on when everything's wrapped up, it may or may not make the 8.0 release; if not, then it will be part of whichever release is upcoming. So no, I don't think you need to worry about release procedures for this issue. Is there something specific you're concerned about?

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Hi Steve, thank you for the quick response.

Is there something specific you're concerned about?

No, I am just wondering that there is something I have to consider. I will create a pull request for review after the next release, Luke 7.6.0. (or 8.0.0?) :)

asfimport commented 6 years ago

Steven Rowe (@sarowe) (migrated from JIRA)

I will create a pull request for review after the next release, Luke 7.6.0. (or 8.0.0?)

Please don't wait for a release; Lucene/Solr development happens independently from release cycles.

asfimport commented 6 years ago

Tomoko Uchida (@mocobeta) (migrated from JIRA)

Please don't wait for a release; Lucene/Solr development happens independently from release cycles.

I understand, thanks!