Provide a "real" Solr Plugin Directory

tboeghk commented 4 years ago

Rafal did a great write up on the new Solr Plugin system and how to provide a plugin directory: https://sematext.com/blog/solr-plugins-system/

Setup repo.solr.cool
Add linked plugins if still available
Provide docs how to use in Solr 8.4+

epugh commented 4 years ago

If you were able to get this rolling, I'd be happy to contribute towards updating various plugins to work with the new Solr Plugin stuff!

otisg commented 4 years ago

💯 for this. Was just thinking how it would be nice if somebody took some of the Sematext "plugins" and, if they still make sense for Solr 8.x+, turn them into proper Solr plugins. I was just looking at solr.cool and realized that, while it's nice to have a nice place for various plugins, there are too many plugins with build errors.

epugh commented 4 years ago

I wonder if a pivot to solr.cool being specifically for Solr 8+ plugins? Instead of a collection of random projects? Or, maybe just put a tag that "plugin_compatible"...?

tboeghk commented 4 years ago

I like things how things are starting here ;-)

As solr.cool is just a catalogue of plugins, I wanted to check out, if I could generate some kind of meta catalogue for plugins without actually hosting (a.k.a downloading) them. I quickly implemented a local version of Rafals repo and hook it up to a Solr. And it works!

So my idea is that we build a repo.solr.cool that is directly usable from within Solr. As the second step, we generate the solr.cool website from the repository.json metadata of the repo.solr.cool website.

More random ideas:

We do not host the binaries, we just link to them on Maven Central. We have to sign them though.
What's the best way to check, collect and sign binaries. It is certainly doable with a Maven POM, but how can we keep it maintainable?
A lot of the plugins listed on solr.cool are vanished or seem fairly unmaintained. We should keep them listed for reference (as there seems to be no other place)

I'll hack more if this together and think about some kind of automation over the next days.

gr0 commented 4 years ago

@tboeghk count me in. I can help with updating some of the extensions and turning them into something usable for the Solr plugins mechanism.

tboeghk commented 4 years ago

I hacked things pretty rough in https://github.com/solr-extensions/repo.solr.cool.

Major point missing here are

verify the integrity of the downloaded jar
actually working plugins ;-)
CI/CD pipeline to build and test PRs
a simple test whether the listed plugins are installable and removable from a running Solr instance.

I'll add some docs within the next days

renekrie commented 4 years ago

@tboeghk Related to your earlier comment: artefacts on Maven Central are already signed. Do we need an additional signature for plugins?

epugh commented 4 years ago

Very exciting. Having some examples and a central website will encourage folks to embrace this!

gr0 commented 4 years ago

Awesome @tboeghk 👍

tboeghk commented 4 years ago

@tboeghk Related to your earlier comment: artefacts on Maven Central are already signed. Do we need an additional signature for plugins?

Yes, we need to sign artifacts with the repo distribution key. But I‘ll add signature verification for maven central artifacts later!

dsmiley commented 4 years ago

This is really awesome! I'd like to see the Solr Ref Guide and information from Solr project committers (e.g. me) pimping solr.cool as much as possible as the go-to place to find 3rd party Solr plugins. It seems the repo.solr.cool spot is a great start but not ready to share widely. When it is ready, please share that status here. Also please share feedback to the Solr project to help make the package management system better as you see issues.

BTW I've noticed mixed/jumbled language around referring to the new system. Solr's "Package Manager" or "Package management system" are both good ways to refer to it. It doesn't manage plugins themselves, it manages packages (of plugins). And although "plugin" and "extension" are probably synonyms, let's please just refer to them as "plugins".

gus-asf commented 4 years ago

Do you plan to only list things that are plugging into solr, or also "related tools" like https://plugins.gradle.org/plugin/com.needhamsoftware.solr-gradle

epugh commented 4 years ago

We have a list of utilities under https://solr.cool/#utilities, so I don't see why this couldn't be added. The value in http://solr.cool goes up the more projects are included!

tboeghk commented 4 years ago

@gus-asf we definitely will! Can you open a separate issue and/or PR?

tboeghk commented 4 years ago

In a first step I implemented a test harness and added the thymeleaf templating package as test bed in #34

solr-cool / solr-cool.github.io

Provide a "real" Solr Plugin Directory #16