search

wmde / wikibase-release-pipeline

BSD 3-Clause "New" or "Revised" License
44 stars 31 forks source link

WikibaseCirrusSearch doesn't seem to be working with the Docker files shown as examples #383

Open rodrigomorales1 opened 1 year ago

rodrigomorales1 commented 1 year ago

The context

I'm running Wikibase in Docker locally for experimentation purposes. I'm using the files from the main branch as of the time of this writing (permalink to commit). Namely, the files shown below.

I've run the following command to start Docker

$ docker-compose -f docker-compose.yml -f docker-compose.extra.yml up

and this is the state of the containers

$ docker container ls
CONTAINER ID   IMAGE                                    COMMAND                   CREATED        STATUS        PORTS                                   NAMES
54c9604c8d77   wikibase/quickstatements:wmde.9          "/bin/bash /entrypoi…"    10 hours ago   Up 10 hours   0.0.0.0:8840->80/tcp, :::8840->80/tcp   wikibase-docker-quickstatements-1
325cad783991   wikibase/wdqs:0.3.115-wmde.9             "/entrypoint.sh /run…"    10 hours ago   Up 10 hours                                           wikibase-docker-wdqs-updater-1
e922710ef662   wikibase/wdqs-frontend:wmde.9            "/entrypoint.sh ngin…"    10 hours ago   Up 10 hours   0.0.0.0:8834->80/tcp, :::8834->80/tcp   wikibase-docker-wdqs-frontend-1
0a0f68a3ddad   wikibase/wikibase-bundle:1.37.6-wmde.9   "/bin/bash /entrypoi…"    10 hours ago   Up 10 hours   0.0.0.0:80->80/tcp, :::80->80/tcp       wikibase-docker-wikibase-1
abee40ac8f18   wikibase/wikibase-bundle:1.37.6-wmde.9   "/bin/bash /jobrunne…"    10 hours ago   Up 10 hours   80/tcp                                  wikibase-docker-wikibase_jobrunner-1
8eaf006bc8c9   wikibase/wdqs-proxy:wmde.9               "/bin/sh -c \"/entryp…"   10 hours ago   Up 10 hours   80/tcp                                  wikibase-docker-wdqs-proxy-1
33c83053b5ac   wikibase/elasticsearch:6.8.23-wmde.9     "/usr/local/bin/dock…"    10 hours ago   Up 8 hours    9200/tcp, 9300/tcp                      wikibase-docker-elasticsearch-1
9b10e5d0bd66   mariadb:10.9                             "docker-entrypoint.s…"    10 hours ago   Up 10 hours   3306/tcp                                wikibase-docker-mysql-1
8aa39a88b1e7   wikibase/wdqs:0.3.115-wmde.9             "/entrypoint.sh /run…"    10 hours ago   Up 10 hours   9999/tcp                                wikibase-docker-wdqs-1

The problem

If I search insource:"has been", the page Main Page is the only result shown, because it contains the sentence MediaWiki has been installed.. However, if I search haswbstatement:*, no result is shown and the message There were no results matching the query. is shown even though I have Wikibase items that have statements.

This is the URL that was shown in the top of my web browser after I pressed the button Search (see screenshot below).

image

Additional information

I found this issue (archive) published in the Gitlab instance (https://gitlab.opensourceecology.de). A user mentioned that haswbstatement:* showed 0 results, which is the same problem that I'm describing here. @lucaswerkmeister shared a link which, apparently, implied that the issue had been solved. I tried visiting the same endpoint in my instance and this was the result. I'm including it below just in case it helps to find the root cause of this issue.

$ curl 'http://localhost/wiki?title=Item:Q4&action=cirrusDump'
[{"_index":"my_wiki_content_first","_type":"page","_id":"11","_version":[],"_source":{"version":736,"wiki":"my_wiki","namespace":120,"namespace_text":"Item","title":"Q4","timestamp":"2022-11-20T04:42:53Z","create_timestamp":"2022-11-19T18:25:39Z","redirect":[],"incoming_links":0,"category":[],"external_link":[],"outgoing_link":["Item:Q162","Property:P1","Property:P3"],"template":[],"text":"research\nsystematic study undertaken to increase knowledge","source_text":"research\nsystematic study undertaken to increase knowledge","text_bytes":1073,"content_model":"wikibase-item","label_count":1,"labels":{"en":["research"]},"labels_all":null,"descriptions":{"en":"systematic study undertaken to increase knowledge"},"statement_keywords":[],"statement_count":2,"sitelink_count":0,"display_title":"research (Q4)"}}]

Here's the output shown above pretty-printed.

$ curl 'http://localhost/wiki?title=Item:Q4&action=cirrusDump' | jq
[
  {
    "_index": "my_wiki_content_first",
    "_type": "page",
    "_id": "11",
    "_version": [],
    "_source": {
      "version": 736,
      "wiki": "my_wiki",
      "namespace": 120,
      "namespace_text": "Item",
      "title": "Q4",
      "timestamp": "2022-11-20T04:42:53Z",
      "create_timestamp": "2022-11-19T18:25:39Z",
      "redirect": [],
      "incoming_links": 0,
      "category": [],
      "external_link": [],
      "outgoing_link": [
        "Item:Q162",
        "Property:P1",
        "Property:P3"
      ],
      "template": [],
      "text": "research\nsystematic study undertaken to increase knowledge",
      "source_text": "research\nsystematic study undertaken to increase knowledge",
      "text_bytes": 1073,
      "content_model": "wikibase-item",
      "label_count": 1,
      "labels": {
        "en": [
          "research"
        ]
      },
      "labels_all": null,
      "descriptions": {
        "en": "systematic study undertaken to increase knowledge"
      },
      "statement_keywords": [],
      "statement_count": 2,
      "sitelink_count": 0,
      "display_title": "research (Q4)"
    }
  }
]

Personal thoughts

I think the problem here is the extension WikibaseCirrusSearch. I'd appreciate anyone could help me to find the root cause of this issue. If you need more information of my system, please let me know.

rodrigomorales1 commented 1 year ago

In this Github Gist , you can find the output of executing docker container logs in the container wikibase/elasticsearch:6.8.23-wmde.9.

$ docker container logs 33c83053b5ac

For the record, this is the output of docker container ls.

$ docker container ls      
CONTAINER ID   IMAGE                                    COMMAND                   CREATED        STATUS        PORTS                                   NAMES
54c9604c8d77   wikibase/quickstatements:wmde.9          "/bin/bash /entrypoi…"    11 hours ago   Up 11 hours   0.0.0.0:8840->80/tcp, :::8840->80/tcp   wikibase-docker-quickstatements-1
325cad783991   wikibase/wdqs:0.3.115-wmde.9             "/entrypoint.sh /run…"    11 hours ago   Up 11 hours                                           wikibase-docker-wdqs-updater-1
e922710ef662   wikibase/wdqs-frontend:wmde.9            "/entrypoint.sh ngin…"    11 hours ago   Up 11 hours   0.0.0.0:8834->80/tcp, :::8834->80/tcp   wikibase-docker-wdqs-frontend-1
0a0f68a3ddad   wikibase/wikibase-bundle:1.37.6-wmde.9   "/bin/bash /entrypoi…"    11 hours ago   Up 11 hours   0.0.0.0:80->80/tcp, :::80->80/tcp       wikibase-docker-wikibase-1
abee40ac8f18   wikibase/wikibase-bundle:1.37.6-wmde.9   "/bin/bash /jobrunne…"    11 hours ago   Up 11 hours   80/tcp                                  wikibase-docker-wikibase_jobrunner-1
8eaf006bc8c9   wikibase/wdqs-proxy:wmde.9               "/bin/sh -c \"/entryp…"   11 hours ago   Up 11 hours   80/tcp                                  wikibase-docker-wdqs-proxy-1
33c83053b5ac   wikibase/elasticsearch:6.8.23-wmde.9     "/usr/local/bin/dock…"    11 hours ago   Up 8 hours    9200/tcp, 9300/tcp                      wikibase-docker-elasticsearch-1
9b10e5d0bd66   mariadb:10.9                             "docker-entrypoint.s…"    11 hours ago   Up 11 hours   3306/tcp                                wikibase-docker-mysql-1
8aa39a88b1e7   wikibase/wdqs:0.3.115-wmde.9             "/entrypoint.sh /run…"    11 hours ago   Up 11 hours   9999/tcp                                wikibase-docker-wdqs-1
schelv commented 1 year ago

@rdrg109 statements are not indexed by default. You need to add this configuration option (with the identifiers of the properties you want indexed): $wgWBRepoSettings['searchIndexProperties'] = ['P1', 'P2'];

Make sure that it's also mounted into the jobrunner since that container is doing the actual indexing. (see https://github.com/wmde/wikibase-release-pipeline/pull/390/files)