SychO9
commented
11 months ago Update
Filtering vs Searching
My initial line of thought for having no separation at all between simple filtering and searching was incorrect. It would be sane to keep on relying on the database implementation when filtering only, and in the future we might easily introduce the ability to change that behavior per driver if it makes sense to do so.
An upside is that this only means w default to the database search driver for that.
Indexing
I initially wanted to leave the logic for listening to events and indexing models to the driver extension implementation. But fter looking into a POC it is clear to me that there is enough repetitive boilerplate involved that it would make sense to expose an API for it that makes it more seamless.
- The indexing operations would be called from within a queueable job.
- Indexers would not be driver specific, meaning you could index a model without having a driver searcher for it. This is because while you might have a discussion driver searcher, you may not have a post driver searcher implemented, and yet the former would require indexing both discussions and posts.
The only ting I'm unsure of at the moment is if we should also have a built in index building command or whether to leave that to extensions. Since we would have the indexer classes, then we could build the command, but extensions might require custom commands with custom options regardless so I'm leaning towards leaving it up to extensions.
(new Extend\SearchIndex)
->indexer(Discussion::class, DiscussionIndexer::class)
->indexer(Post::class, PostIndexer::class),interface IndexerInterface
{
static function index(): string;
/**
* @param AbstractModel[] $models
*/
function save(array $models): void;
/**
* @param AbstractModel[] $models
*/
function delete(array $models): void;
}class ModelObserver
{
public function __construct(
protected SearchManager $search,
protected Queue $queue
) {
}
public function created(AbstractModel $model): void
{
$this->runIndexJob($model, 'save');
}
public function updated(AbstractModel $model): void
{
$this->runIndexJob($model, 'save');
}
public function hidden(AbstractModel $model): void
{
$this->runIndexJob($model, 'delete');
}
public function deleted(AbstractModel $model): void
{
$this->runIndexJob($model, 'delete');
}
public function forceDeleted(AbstractModel $model): void
{
$this->runIndexJob($model, 'delete');
}
public function restored(AbstractModel $model): void
{
$this->runIndexJob($model, 'save');
}
private function runIndexJob(AbstractModel $model, string $operation): void
{
if ($this->search->indexable($model)) {
foreach ($this->search->indexers($model) as $indexerClass) {
$this->queue->push(new IndexJob($indexerClass, [$model], $operation));
}
}
}
}
This is a breakdown of the current implementation of searching/filtering, and a plan to implement search drivers. The work done prior to stable 1.0 presents a good base to build on top. But there is some refactoring to be done first.
Current Implementation
The current system in place separates the concept of searching & filtering. So we have Gambits and Filters, which are in practice both filtering mechanisms that do the the same thing. This adds some complexity on the backend side as well as the frontend side where applying a filter (like when clicking on the following sidebar link to show subscribed to discussions) requires doing it in two formats by checking if a search is active or not.
The choice between filtering and searching rests on whether a
qfilter is active. From there we either apply basic database filtering, or searching.The current
AbstractSearchers are a hardcoded database search driver implementation.What should not change:
Current search extensions
Brainstorming
Gambits Vs Filters
Dropping gambits on the backend should be the first step. Gambits are just filters apply in a different format. We can transform gambits to API request filters from the frontend. The additional advantage of that is the frontend becomes aware of gambits which allows us to improve the search UI/UX after the work here is done.
Filterers VS Searchers
Removing gambits still means we need to have filters per searcher. We could simply replace the idea of gambits in searchers with filters, but that leaves some unnecessary duplication between Filterers Vs Searchers.
It seems clear that we can simply do without the separate concept of a Filterer. If a search driver implements its own filtering (like Blomstra's elastic implementation) then it can do the filtering. If a driver does not (like Clark's scout) then it can fallback to the database driver to apply filters (like Clark's scout impl). The difference lies in what the driver does. So it needs to be able to fallback to the database driver after it has the results it needs (the same line of thinking applies to visibility scoping).
Implementing drivers
The rest should be fairly straightforward to implement after the above is taken care of.
What we need is:
searchand index saving/index deletion which will be called internally when appropriate.I envision the use from the controller to be something along the lines of