Compatibility with eZ Platform where SEARCH_ENGINE is Solr

[alexwilson]

Hey,

I'm aware that there have been efforts made to allow this bundle to work with Solr, however with transactions enabled nearly all successive operations to fail as every operation is run through the Search Service.

A short-term remedy for this would be to replace the QueryBasedMatcher with something allowing fixed scalar values to reference content/location/etc IDs directly. This would allow references to continue to work, without waiting for repository updates to be committed to Solr.

More long-term, it's worth noting that the Solr and ElasticSearch search engines actually expose a design feature with the SearchService: It is asynchronous and is not intended for chaining together successive operations in a commit, it is designed for content searches.

Most of the time, when we already have a reference to an ID or Remote ID for a given item, we should be able to use that value directly, without running it through a Query.

[gggeek]

Hello

while we could indeed skip usage of the SearchService for all accesses done directly via id or remote_id, the door would still be left wide open for end users to shoot themselves in the foot: some migrations would be working always well, some would be working well only when run standalone and not next to each other. And it would be hard to tell why, even for experienced eZ developers. Also, since we would still need to support the Query-based matching for all complex cases, we would have to maintain double code paths, which increases the maintenance and testing burden.

I think that we should be looking directly for the better, "global" solution.

We could start f.e. with a big warning given to end users whenever the solr-based search service is found and the migration attempts to use it for matching content/location.

Another approach would be trying to allow, via a migration config or a cli switch, to grab a handle to the db-search-service even when the solr-search-service is in use. Note that I am not sure if this is even possible, I seem to recall from eZ engineers that the kernel only has room for one search service at a time...

Any other approach proposition is also welcome, of course

[alexwilson]

Hey

Pinged you on Slack as I thought that a more "real-time" conversation at some point might be useful, but, maybe we can hash-out a quick solution here!

Completely agreed that this would still allow people running with a non-Legacy Search Engine implementation to shoot themselves in the foot, but maybe that's something avoidable by briefly documenting some "best practices"?

I've been thinking that pluggable Matchers might make sense? You can opt-in through configuration to use that, which would use Repository services where possible, for identifiers.

What do you think?

[gggeek]

I have implemented your PR, and added some info about this situation in the main README file.

I also have added a 'sleep' migration step, which might be useful to make sure that different mig steps can be executed separated by a time interval. This could conceivably be used to avoid problems related to Solr (or even db transactions) using the following strategy:

• separate original migration in different mig files
• run migration command(s) separately so that each migration is run in a transaction, but without a global transactions spanning the whole thing. Easily achieved by using the -p flag
• separate execution of the different migrations by adding in sleep steps in migrations 2 .. N

[gggeek]

Ps: current fixes are released as part of 4.3. Not sure if we should close this ticket or keep it open for future memory...