Open ygale opened 5 years ago
jvolkenant
commented
5 years ago bionic has built in packages for solr-tomcat (and solr-jetty). That would be preferred over manually installing tomcat. There needs to be some testing (and PR) since some new packages are now available in apt. I would like to see FTS again in bionic, but I can't verify what clients actually use FTS vs searching locally.
ygale
commented
5 years ago bionic is on Solr 3.6.2. It is true that solr-tomcat and solr-jetty are the standard ways to run Solr on bionic. It is also possible to install only solr-common and run Solr directly, without tomcat or jetty. In that case, Solr launches its own jetty internally.
The direct way is not the normal way to launch Solr in bionic. But in more recent versions of Solr, the direct way is the only supported way to launch Solr. Running Solr in a separate tomcat or jetty is deprecated starting in Solr 5. Here is the explanation of why.
So we would reduce later upgrade friction if we just run it directly rather than using solr-tomcat or solr-jetty. I'm not sure it would be any harder, either; it's not like configuring tomcat is totally trivial. The only question is whether we can re-use the original work in #251, which used tomcat, and if so, whether it is worth it to push off that change until the next Ubuntu upgrade.
I can't verify what clients actually use FTS vs searching locally
What does "searching locally" mean?
jvolkenant
commented
5 years ago Built in packages would be the easiest method to maintain. Would we really be missing much using solr from apt, vs downloading and installing the latest version manually?
By searching locally, I mean, it's difficult to verify which email clients (Thunderbird, smtp ones on Android, etc.) actually use FTS, or if they just use an index locally within the application to do searching. Some clients don't specify if they support FTS or not. Are we reaching the point to where the available clients don't have FTS support?
ygale
commented
5 years ago Oh yes, agreed, we should definitely use packages and not install manually. I think there are two options how to do that though: (1) install dovecot-solr, solr-tomcat, and solr-common, and then configure tomcat; or (2) install only dovecot-solr and solr-common, and then configure the init scripts to launch solr.
(1) is more complicated (I think), and it is deprecated in Solr so it will put us back into this same situation for the next Ubuntu upgrade. (2) is a less standard approach for bionic, but will become the standard approach in future Ubuntu releases.
Local searching by clients - ah OK. You're right, I don't know either which local clients if any use server-side FTS. I was talking about the web interface.
JoshData
commented
5 years ago Just FYI, I'm planning to push out the ubuntu_bionic branch as soon as it's stable --- so we will lose FTS when it is released. Adding Solr back in is something I'm open to, but I can't let it be a blocker, and I'm not sure I'm going to have the time to review any changes related to adding it any time soon.
ygale
commented
5 years ago @JoshData Thanks! Absolutely makes sense to get bionic-based MIAB out the door. We can keeping working on FTS.
jvolkenant
commented
5 years ago
I propose we reconsider adding FTS based on dovecot-solr.
Brief history:
PR #251 originally added FTS using dovecot-solr. That ultimately evolved to using dovecot-lucene instead. A brief summary of the reasons for the switch is in #982, and in more detail in the discussion in #251. But because dovecot-lucene has long been deprecated, it is increasingly impractical to use, and currently it looks like the new bionic-based MIAB may not have FTS at all because of that.
In the modern era, search is no longer just nice to have. It has become part of common culture, and it is expected. Today, lack of FTS support would be a major hit on the relevance of MIAB, and hence on its ability to contribute its goodness.
The only currently supported FTS plugins in dovecot are solr, and a new plugin only available in the commercial version of dovecot. So if we are using dovecot, it seems dovecot-solr is the only reasonable option.
The dovecot-solr package is available in bionic. #251 already went a long way toward integrating it nicely into MIAB. It does add a dependence on Java, and there are a few security points that need to be covered, but these are all doable, as seen in #251. Let's go for it.