Documents inserted into ElasticSearch has a query-property

[fangel]

I'm currently looking into some improvements that we want to do on our metadata-search, so I was looking into the raw data that got stored in ElasticSearch, and I noticed something strange. All of the documents include the following

...,
"mime": "...",
"query": {
  "filtered": {
    "filter": {
      "and": []
    }
  }
},
"size": ...,
...

That seemed a little suspicious, and like a artefact of some search-query. So I looked around a bit, and have figured out it's because the same function, prepareParams is called from both set and search, and by default it populates a default search-query for the search functionality so that the later functions can always just extend that filter.

if (isset($params['body']['query']['filtered']['filter'][0])) {
  ...
} else {
  $params['body']['query']['filtered']['filter'] = ['and' => []];
}

But this is has the side-effect of also injecting the default search query into all the documents that are stored, which is probably not ideal.

I can take a look at fixing this, if you want - I would probably just move the query-part of prepareParams into search...

[kbrabrand]

Sounds good 👍 I'm not currently using Imbo much, and have little time left to contribute 😢

[fangel]

Wonderful. I'll take a look at it then.

There'll probably be a few others related to full-text searches, as some of our users have complained that the ordering of results isn't ideal. But I'll send them in so you can take a look at them when we get that far.

Have you left VG since you aren't Imbo much anymore?

[kbrabrand]

👍

Yes – both me and @rexxars left VG a little more than a year ago. Christer is still sort of there, even though he's in Schibsted now.

[fangel]

Oh, okay. We were considering a trip up to visit VG sometime in the future to hear what you guys are up to - but if most of you guys aren't there anymore it might not make as much sense.

So where are you now, if I might ask?

[kbrabrand]

Aight. I would talk to @christeredvartsen about that.. I don't know what the plans for Imbo @ VG are going forward, but there might be more suitable channels than this issue for discussing that 🙈

I'm in Netlife Research and Espen is working for Bengler.

[christeredvartsen]

@fangel I'm still working for VG although I'm hired by Schibsted. :)

[fangel]

Thanks for the info. Christer, we might get in contact with your about a potential visit sometime in the future.

And sorry for derailing my own issue - I just got curious.