Usage: combining queries

[myselfhimself]

Hello,

Thank you again very much for the Larasearch library.

I would like to query for documents having altogether :

• given keywords within a set of analyzed fields,
• some_field = NN,
• some_other_field = NN
• .. etc.

The first bullet is doable natively by following the examples in README.md.

For combining with other bullets it does not seem so. Looking at Query->getPayload() which is one of Larasearch's last step before telling the ES PHP Client to search, it does not seem feasible for now to have more queries within the dismax handler than the existing series of lookup for $this->term within the given analyzed fields.

How is that possible to search for multiple terms in different fields in Larasearch ?

[iverberk]

Thanks for the feedback. This definitely needs to be added. I have to think a little bit on the nicest interface but I'll try to implement something this week. Any suggestions on how you would like to see this implemented from a search point of view are welcome :-)

[myselfhimself]

Hello, I will travel to Germany tonight for 9 months to do something than else than software development, so @menthol will continue discussing this ticket with you.

Documenting direct use of ES client library with credentials injected

Larasearch leverages the official elasticsearch library. I have taken a look at http://www.elasticsearch.org/guide/en/elasticsearch/client/php-api/master/_search_operations.html and found that building a search query array is almost transparent to building the related json payload to be sent. I am sure that the Larasearch project will always have a little lag (this is not pejorative) behind the elasticsearch PHP library or Elasticsearch's server features. It would be nice to document in the README.md how it is possible to bypass Eloquent/Larasearch search()method and build and run an Elasticsearch query with automatic dependency injection of the target hosts details. Eg. `Larasearch::client()->search($phpElasticSearchClientLibFormattedParamsArray);

Multi-queries : basic proposal

Inspiration

Searchkick which inspired your project still has Search multiple fields for different terms in their roadmap at this date.

Similarly Elasticsearch-Rails, makes its repository.search method accept Lucene QueryParser DSL (handier for simple usages, including mutliple terms-fields ones) or ES JSON-like DSL.

Respectively (Query String Query / Lucene DSL, ES DSL) :

repository.search('fox or dog').to_a
# GET http://localhost:9200/notes_development/my_note/_search?q=fox
# => [<MyNote ... FOX ...>, <MyNote ... DOG ...>]

repository.search(query: { match: { title: 'fox dog' } }).to_a
# GET http://localhost:9200/notes_development/my_note/_search
# > {"query":{"match":{"title":"fox dog"}}}
# => [<MyNote ... FOX ...>, <MyNote ... DOG ...>]

Here is a ES JSON DSL example application.rb search call with Aggregations, Highlighting...

Elastica has classes for all kinds of kinds of queries, and Larasearch could well have chainable/boolean-op-joinable classes for building Queries before injecting them into a search() method. I do not think the objective is to replace the library in use, though.

Early proposal

Having a SomeModel::search() Eloquent way of querying is neat in itself. Here is a minimalistic API proposal, imitating a little Elasticsearch-rails application.:

// Function prototype
// $query => string DSL query, (?non-DSL string) or JSON-like DSL query array
// $params => example keys : aggs, sort, size, fields (? for non-DSL string query only), highlight...
Eloquent::search($string|$array query, $params);
// Allow for Query String Query natively
Eloquent::search('(content:this OR content:thus) AND (name:this OR name:thus)')
// Allow for search query arrays
Eloquent::search(  [ "match" => ["text" => q]], $params);

Just ideas...

[myselfhimself]

Other inspiration: Elasticquent has a Model::searchQuery method that accepts an ES JSON-like array as first parameter.

[iverberk]

@myselfhimself: thanks for your feedback, good luck in Germany!

@menthol: This is an omission in the documentation but it is in fact already possible to pass an elasticsearch-php compatible array or json structure directly to the Elasticsearch client. I will add this to the doc as soon as possible. I just added a little bugfix on develop so pull that in first before trying the examples below. Example use cases:

$query['query']['match']['name'] = 'Raymundo';
$husband = Husband::search(null, ['query' => $query])->getResults();

$wife = Wife::search(null, ['json' => '{"query": {"match": {"name": "Lisa"}}}'])->getResults();

This gives you total freedom in constructing the queries. Somewhere there is a sweetspot to abstract some of this functionality into Larasearch, but it will never be as complete as the native client possibilities.

Searchkick has a 'where' option that you can pass to the search query to define filters. I think this might be appropriate, e.g.: Product::search("term", ['where' => ['in_stock' => true]]

[myselfhimself]

Thanks @iverberk , your reply gets us unstucked!!!!

[myselfhimself]

a 'where' parameter would be nice, it could be tuned with boolean operators...

[iverberk]

I'll try to implement similar functionality soon. I've added a new section to the readme on free-form searching.

[iverberk]

Did you use JobAd::search(null, ['query' => $params])? This should get rid of all the other parameters and use the literal array as input for the Elasticsearch client, except for some pagination parameters.

[myselfhimself]

Thanks !

Here's another version that does not work :

        $params = [];
        // @todo change nb JobAds per page
        // @todo paginate results
        $params['query']['filtered']['query'] = [
            'multi_match' => array(
                'query' => 'assistant',
                'type' => 'best_fields',
                'fields' => array('title.analyzed^2', 'job_title.male_name.analyzed', 'job_title.female_name.analyzed'),
                'tie_breaker' => 0.3,
                'minimum_should_match' => '30%',
            )
        ];

        $params['query']['filtered']['filter'] = [
            'bool' => array(
                'must' => array(
                    'term' => ['status' => JobApplication::STATUS_CREATED]
                ),
            )
        ];

      $params['aggs']['location'] = ['type' => 'terms', 'field' => 'address.city', 'order' => ["_term" => "asc"]];
        $params['aggs']['status'] = ['type' => 'terms', 'field' => 'status', 'order' => ["_term" => "asc"]];
        $params['aggs']['job_title'] = ['type' => 'terms', 'field' => 'job_title.id', 'order' => ["_term" => "asc"]];

        $jobAdsResponse = JobAd::search(null, $params);

{"error":"SearchPhaseExecutionException[Failed to execute phase [query_fetch], all shards failed; shardFailures {[OFJp2SyHSrenBaqlLMJdzA][job_ads_20140930143312][0]: SearchParseException[[job_ads_20140930143312][0]: from[0],size[50]: Parse Failure [Failed to parse source [{\"size\":50,\"from\":0,\"aggs\":{\"location\":{\"terms\":{\"field\":\"address.city\",\"size\":0}},\"status\":{\"terms\":{\"field\":\"status\",\"size\":0}},\"job_title\":{\"terms\":{\"field\":\"job_title.id\",\"size\":0}}},\"filtered\":{\"query\":{\"multi_match\":{\"query\":\"assistant\",\"type\":\"best_fields\",\"fields\":[\"title.analyzed^2\",\"job_title.male_name.analyzed\",\"job_title.female_name.analyzed\"],\"tie_breaker\":0.3,\"minimum_should_match\":\"30%\"}},\"filter\":{\"bool\":{\"must\":{\"term\":{\"status\":0}}}}}}]]]; nested: SearchParseException[[job_ads_20140930143312][0]: from[0],size[50]: Parse Failure [No parser for element [filtered]]]; }]","status":400}

To circumvent that kind of errors, I have created myself a Model::searchBare($options) method that skips puts all the $options (except 'index' and 'type' keys) inside the request's body...

[myselfhimself]

I will make a pull request to propose my Model::searchBare($options) method; I think the name is not a good one... It is not set that I will work 2 more days within the following 2 weeks around larasearch & ES. We may get to talk together again. Thank you for your great work and open-sourcing.

[myselfhimself]

That bareSearch patch does not fix anything obviously: Model::search(null, ['query' => $params]) does work flawlessly and I am not using that bareSearch($params) method anymore. Sorry for bothering you with that. It would be nice to have a simpler method signature for bare $params passing though.

[iverberk]

Agreed, I will see if I can come up with something. Thanks for your constructive feedback though.

[smithdp1]

actually this works for me.

$params['match']['YOUR_COLUMN'] = $YOURVAR;
$results = YOUR_MODEL::search(\Input::get('q', '*'), $params);