Query Behaviour changed to include id prefix inside query

[Koalabaerchen]

I ran https://github.com/floriansemm/SolrBundle/commit/37865d45bccebc3698b8f911d9d459fc478bc453 before upgrading today since the 1.6 (https://github.com/floriansemm/SolrBundle/commit/ea957fcf483afd951ba109029a2f4bb509c6d4b8) got released. Since then, my tests (and application) fail due to a changed query.

I have two entities that populate Solr: Company and Job. I'm doing an autosuggest functionality on the Job entity. So I do a query on all the fields (add them all with $query->addSearchTerm($field, $search);). I'm using HYDRATEINDEX with a filterQuery onto the ids starting with job*

Since the update it adds the id field prefix to all queries and puts the rest with "AND" behind it.

Before the query was select?omitHeader=true&wt=json&json.nl=flat&q=title_t:render OR company_txt:render OR occupational_fields_txt:render OR employment_types_txt:render OR degree_levels_txt:render OR fields_of_study_txt:render OR skills_txt:render&start=0&rows=10&fl=&sort=title_t asc&fq=id:job_* Now it's select?omitHeader=true&wt=json&json.nl=flat&q=id:job_* AND title_t:render OR company_txt:render OR occupational_fields_txt:render OR employment_types_txt:render OR degree_levels_txt:render OR fields_of_study_txt:render OR skills_txt:render&start=0&rows=10&fl=&sort=title_t asc&fq=id:job_*

The difference is the id:job_* AND it adds to the beginning of the query (q=). Apparently the boolean logic is wrong there. It checks if the id starts with job_ AND if the title_t includes "render". If that doesn't happen (because it is in one of the other fields), it just returns an empty result.

From what I understand of Solr the id:job_* belongs into the FilterQuery (fq), not into the query itself?

[floriansemm]

Maybe an Solarium issue? Between 1.5.5 and 1.6 no changes were made in the query component.

[floriansemm]

Can you check your Solarium version?

[Koalabaerchen]

Solarium version didn't change, only thing that changed (to get it to work again) is

Loading composer repositories with package information
Installing dependencies (including require-dev) from lock file
  - Removing minimalcode/search (1.0.0-RC1)
  - Updating floriansemm/solr-bundle (v1.6 => dev-master 37865d4)
    Checking out 37865d45bccebc3698b8f911d9d459fc478bc453

Generating autoload files

Solarium is at 3.6.0, which is the required version.

[floriansemm]

Can you show me the code of your query?

[Koalabaerchen]

Had to remove some logic that isn't necessary for this query and just makes things more complicated

    // Create new Solr query and get the search param
    $query = $this->get('solr.client')->createQuery(new Job());
    $search = $request->get('search');

    $query->addSearchTerm('title', $search);
    $query->addSearchTerm('occupationalFields', $search);

    // Filter by id to search only either jobs or companies, not both
    $fq = $query->createFilterQuery('id')->setQuery("id:job_*");
    $query->addFilterQuery($fq);

    // Hydrate search results from doctrine to have complete entities and set the amount of results to get
    $query->setHydrationMode(HydrationModes::HYDRATE_INDEX);

    // Get results and format them
    $results = $query->getResult();

$search is the POSTed search query

occupationalFields is ManyToMany title is a string

Log entry for my test on the POST with search = 'render' is [2016-11-02 18:37:59] solr.INFO: run request: select?omitHeader=true&wt=json&json.nl=flat&q=id:job_* AND title_t:render OR occupational_fields_txt:render&start=0&rows=10&fl=&fq=id:job_* [] [] Returns an empty array, although 2 results are expected. The word "render" is in the occupationalFields of both results in this test.

edit: I changed the code snippet to dumb it down more. Now it's basically a basic query. Still the same problem. I even removed the filterQuery, Nothing changed

[floriansemm]

You are right: fq should be used for ID queries like id:job_*, I will fix this.

To fix your problem you can remove this lines:

// Filter by id to search only either jobs or companies, not both
$fq = $query->createFilterQuery('id')->setQuery("id:${which}_*");
$query->addFilterQuery($fq);

The final query includes always the id field.

[Koalabaerchen]

Removing my own filter query doesn't help, because it's still the problem of how Solr interprets/groups AND and OR.

$query = $this->get('solr.client')->createQuery(new Job());
$search = $request->get('search');

$query->addSearchTerm('title', $search);
$query->addSearchTerm('occupationalFields', $search);

// Hydrate search results from doctrine to have complete entities and set the amount of results to get
$query->setHydrationMode(HydrationModes::HYDRATE_INDEX);

// Get results and format them
$results = $query->getResult();

This doesn't return a result if the searched term is not in the title field but in the occupationalFields field. If I switch the order of the addTerms() around, I get the expected result.

[floriansemm]

release 1.6.1 should fix your problem

[Koalabaerchen]

After I removed my own fqs, it seems to work as intended.

Thanks!