I did an offline import of en.wikipedia.org around 4/24/2019 and all seemed well. Then I repeated the process for en.witionary.org and that seemed to go well too. But I don't recall looking at en.wikipedia.org after that. Now when I click on the en.wikipedia.org link in the GUI, the status bar reports could not find page in wiki: en.wikipedia.org/. The en.wiktionary.org link seems to work fine. This is OpenSuSE 42.3, x86_64. The session log follows. Any hints on how to troubleshoot this? Do all wikis have to be imported in one operation? (Since it has been a couple of months since I did this, I'll probably redo the import procedure since my memory of the events is a bit fuzzy now).

[edited by gnosygnu for formatting]

20190609_180259.290 app.gui.kit_init.done
20190609_180259.376 app.gui.win_load.done
20190609_180259.378 app.launch.page.bgn
20190609_180259.591 app.launch.page.end
20190609_180259.590 page.load: url=home/wiki/Main_Page
20190609_180300.429 page_load: loaded wikitext; page=Main_Page wikitext_len=8743
20190609_180300.497 jtidy.init; elapsed=31
20190609_180300.708 page.async: url=home/wiki/Main_Page
20190609_180300.733 redlink.redlink_bgn: page=Main_Page total_links=36

click en.wikipedia.org link

20190609_180300.775 redlink.redlink_end: redlinks_run=0
20190609_180326.720 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-file-core.xowa;version=3;
20190609_180326.722 wiki.db_core.none: dir=/var/opt/xowa/wiki/en.wikipedia.org/
20190609_180326.695 wiki.init.bgn: en.wikipedia.org
20190609_180326.739 wiki.init.end
20190609_180326.743 page.load: url=en.wikipedia.org/wiki/Main_Page
20190609_180326.756 page_load: loaded wikitext; page=Main_Page wikitext_len=0

click en.wiktionary.org link

20190609_180336.012 wiki.db_core.v2.standard: url=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-core.xowa
20190609_180336.015 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-core.xowa;version=3;
20190609_180336.017 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.004.xowa;version=3;
20190609_180336.019 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.000.xowa;version=3;
20190609_180336.020 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.100.xowa;version=3;
20190609_180336.022 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.012.xowa;version=3;
20190609_180336.024 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.118.xowa;version=3;
20190609_180336.026 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.104.xowa;version=3;
20190609_180336.028 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.008.xowa;version=3;
20190609_180336.030 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.114.xowa;version=3;
20190609_180336.031 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.010.xowa;version=3;
20190609_180336.033 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.102.xowa;version=3;
20190609_180336.034 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.106.xowa;version=3;
20190609_180336.039 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.014.xowa;version=3;
20190609_180336.041 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.110.xowa;version=3;
20190609_180336.043 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.006.xowa;version=3;
20190609_180336.044 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.090.xowa;version=3;
20190609_180336.046 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.092.xowa;version=3;
20190609_180336.052 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.116.xowa;version=3;
20190609_180336.053 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.828.xowa;version=3;
20190609_180336.055 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-text-ns.108.xowa;version=3;
20190609_180336.057 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-xtn.search.core.xowa;version=3;
20190609_180336.059 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-xtn.search.link-title-ns.000-db.001.xowa;version=3;
20190609_180336.060 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-xtn.search.link-title-ns.999-db.001.xowa;version=3;
20190609_180336.062 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-xtn.category.core.xowa;version=3;
20190609_180336.063 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-xtn.category.link-db.001.xowa;version=3;
20190609_180336.065 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-xtn.category.link-db.002.xowa;version=3;
20190609_180336.066 fsdb.db_core.v2: type=lot url=/var/opt/xowa/wiki/en.wiktionary.org/en.wiktionary.org-file-core.xowa
20190609_180336.008 wiki.init.bgn: en.wiktionary.org
20190609_180336.082 wiki.init.end
20190609_180336.086 site_cfg loaded; wiki=en.wiktionary.org itm=extensiontags tid=cfg
20190609_180336.087 site_cfg loaded; wiki=en.wiktionary.org itm=interwikimap tid=cfg
20190609_180336.106 page.load: url=en.wiktionary.org/wiki/Wiktionary:Main Page
20190609_180337.647 wiki.init.bgn: commons.wikimedia.org
20190609_180337.681 wiki.init.end
20190609_180339.244 download pass: src='https://commons.wikimedia.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-us-ennead.ogg' trg='mem/download.tmp'
20190609_180339.244 api failed: <?xml version="1.0"?><api batchcomplete=""><query><pages><page _idx="-1" ns="6" title="File:En-us-ennead.ogg" missing="" imagerepository="" /></pages></query></api>
20190609_180339.799 download pass: src='https://en.wiktionary.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-us-ennead.ogg' trg='mem/download.tmp'
20190609_180339.799 api failed: <?xml version="1.0"?><api batchcomplete=""><query><pages><page _idx="-1" ns="6" title="File:En-us-ennead.ogg" missing="" imagerepository="" /></pages></query></api>
20190609_180340.669 download fail: src='https://wikimedia.org/math/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-us-ennead.ogg' trg='mem/download.tmp' error='https://www.wikimedia.org/math/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-us-ennead.ogg'
20190609_180340.670 download_failed: En-us-ennead.ogg
20190609_180340.832 download pass: src='https://commons.wikimedia.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-uk-ennead.ogg' trg='mem/download.tmp'
20190609_180340.832 api failed: <?xml version="1.0"?><api batchcomplete=""><query><pages><page _idx="-1" ns="6" title="File:En-uk-ennead.ogg" missing="" imagerepository="" /></pages></query></api>
20190609_180340.978 download pass: src='https://en.wiktionary.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-uk-ennead.ogg' trg='mem/download.tmp'
20190609_180340.979 api failed: <?xml version="1.0"?><api batchcomplete=""><query><pages><page _idx="-1" ns="6" title="File:En-uk-ennead.ogg" missing="" imagerepository="" /></pages></query></api>
20190609_180341.368 download fail: src='https://wikimedia.org/math/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-uk-ennead.ogg' trg='mem/download.tmp' error='https://www.wikimedia.org/math/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-uk-ennead.ogg'
20190609_180341.368 download_failed: En-uk-ennead.ogg
20190609_180341.573 download pass: src='https://commons.wikimedia.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-us-ennead.oga' trg='mem/download.tmp'
20190609_180341.574 api failed: <?xml version="1.0"?><api batchcomplete=""><query><pages><page _idx="-1" ns="6" title="File:En-us-ennead.oga" missing="" imagerepository="" /></pages></query></api>
20190609_180341.728 download pass: src='https://en.wiktionary.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-us-ennead.oga' trg='mem/download.tmp'
20190609_180341.728 api failed: <?xml version="1.0"?><api batchcomplete=""><query><pages><page _idx="-1" ns="6" title="File:En-us-ennead.oga" missing="" imagerepository="" /></pages></query></api>
20190609_180342.036 download fail: src='https://wikimedia.org/math/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-us-ennead.oga' trg='mem/download.tmp' error='https://www.wikimedia.org/math/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-us-ennead.oga'
20190609_180342.036 download_failed: En-us-ennead.oga
20190609_180342.191 download pass: src='https://commons.wikimedia.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-uk-ennead.oga' trg='mem/download.tmp'
20190609_180342.191 api failed: <?xml version="1.0"?><api batchcomplete=""><query><pages><page _idx="-1" ns="6" title="File:En-uk-ennead.oga" missing="" imagerepository="" /></pages></query></api>
20190609_180342.353 download pass: src='https://en.wiktionary.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-uk-ennead.oga' trg='mem/download.tmp'
20190609_180342.354 api failed: <?xml version="1.0"?><api batchcomplete=""><query><pages><page _idx="-1" ns="6" title="File:En-uk-ennead.oga" missing="" imagerepository="" /></pages></query></api>
20190609_180342.668 download fail: src='https://wikimedia.org/math/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-uk-ennead.oga' trg='mem/download.tmp' error='https://www.wikimedia.org/math/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-uk-ennead.oga'
20190609_180342.668 download_failed: En-uk-ennead.oga
20190609_180342.825 download pass: src='https://commons.wikimedia.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-au-ennead.ogg' trg='mem/download.tmp'
20190609_180342.825 api failed: <?xml version="1.0"?><api batchcomplete=""><query><pages><page _idx="-1" ns="6" title="File:En-au-ennead.ogg" missing="" imagerepository="" /></pages></query></api>
20190609_180342.971 download pass: src='https://en.wiktionary.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-au-ennead.ogg' trg='mem/download.tmp'
20190609_180342.971 api failed: <?xml version="1.0"?><api batchcomplete=""><query><pages><page _idx="-1" ns="6" title="File:En-au-ennead.ogg" missing="" imagerepository="" /></pages></query></api>
20190609_180343.319 download fail: src='https://wikimedia.org/math/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-au-ennead.ogg' trg='mem/download.tmp' error='https://www.wikimedia.org/math/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:En-au-ennead.ogg'
20190609_180343.326 download_failed: En-au-ennead.ogg
20190609_180343.519 download pass: src='https://commons.wikimedia.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:en-us-ennead.flac' trg='mem/download.tmp'
20190609_180343.519 api failed: <?xml version="1.0"?><api batchcomplete=""><query><normalized><n from="File:en-us-ennead.flac" to="File:En-us-ennead.flac" /></normalized><pages><page _idx="-1" ns="6" title="File:En-us-ennead.flac" missing="" imagerepository="" /></pages></query></api>
20190609_180343.672 download pass: src='https://en.wiktionary.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:en-us-ennead.flac' trg='mem/download.tmp'
20190609_180343.672 api failed: <?xml version="1.0"?><api batchcomplete=""><query><pages><page _idx="-1" ns="6" title="File:en-us-ennead.flac" missing="" imagerepository="" /></pages></query></api>
20190609_180344.191 download fail: src='https://wikimedia.org/math/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:en-us-ennead.flac' trg='mem/download.tmp' error='https://www.wikimedia.org/math/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:en-us-ennead.flac'
20190609_180344.192 download_failed: en-us-ennead.flac
20190609_180344.361 download pass: src='https://commons.wikimedia.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:en-uk-ennead.flac' trg='mem/download.tmp'
20190609_180344.362 api failed: <?xml version="1.0"?><api batchcomplete=""><query><normalized><n from="File:en-uk-ennead.flac" to="File:En-uk-ennead.flac" /></normalized><pages><page _idx="-1" ns="6" title="File:En-uk-ennead.flac" missing="" imagerepository="" /></pages></query></api>
20190609_180344.513 download pass: src='https://en.wiktionary.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:en-uk-ennead.flac' trg='mem/download.tmp'
20190609_180344.514 api failed: <?xml version="1.0"?><api batchcomplete=""><query><pages><page _idx="-1" ns="6" title="File:en-uk-ennead.flac" missing="" imagerepository="" /></pages></query></api>
20190609_180344.831 download fail: src='https://wikimedia.org/math/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:en-uk-ennead.flac' trg='mem/download.tmp' error='https://www.wikimedia.org/math/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:en-uk-ennead.flac'
20190609_180344.831 download_failed: en-uk-ennead.flac
20190609_180348.993 page_load: loaded wikitext; page=Wiktionary:Main_Page wikitext_len=13989
20190609_180349.047 jtidy.init; elapsed=0
20190609_180349.779 wiki.init.bgn: www.wikidata.org
20190609_180349.805 wiki.init.end
20190609_180349.859 page.async: url=en.wiktionary.org/wiki/Wiktionary:Main Page
20190609_180349.891 file.get: file=Writing_star.svg width=62 page=Wiktionary:Main_Page
20190609_180354.520 file.get: file=Wikinews_commentary.svg width=55 page=Wiktionary:Main_Page
20190609_180354.631 file.get: file=Notepad_icon.png width=57 page=Wiktionary:Main_Page
20190609_180354.655 file.get: file=Books-aj.svg_aj_ashton_01e.svg width=64 page=Wiktionary:Main_Page
20190609_180354.658 file.get: file=Wikipedia-logo.png width=35 page=Wiktionary:Main_Page
20190609_180354.671 file.get: file=Wikibooks-logo.png width=35 page=Wiktionary:Main_Page
20190609_180354.674 file.get: file=Wikinews-logo.png width=35 page=Wiktionary:Main_Page
20190609_180354.676 file.get: file=Wikisource-logo.png width=35 page=Wiktionary:Main_Page
20190609_180354.703 file.get: file=Wikispecies-logo.png width=35 page=Wiktionary:Main_Page
20190609_180354.706 file.get: file=Wikiquote-logo.png width=35 page=Wiktionary:Main_Page
20190609_180354.716 file.get: file=Commons-logo.png width=35 page=Wiktionary:Main_Page
20190609_180354.720 file.get: file=Wikimedia_Community_Logo.svg width=35 page=Wiktionary:Main_Page
20190609_180354.723 file.get: file=Wikiversity-logo.svg width=35 page=Wiktionary:Main_Page
20190609_180354.740 file.get: file=Wikivoyage-Logo-v3-icon.svg width=35 page=Wiktionary:Main_Page
20190609_180354.743 file.get: file=Wikidata-logo.svg width=35 page=Wiktionary:Main_Page
20190609_180354.745 redlink.redlink_bgn: page=Wiktionary:Main Page total_links=44
20190609_180355.377 redlink.redlink_end: redlinks_run=0

When xowa run in http_server mode, the browser sees this for http://127.0.0.1/en.wikipedia.org/

[err 0] 2 [trace]: gplx.xowa.apps.servers.http.Http_url_parser.Parse(Unknown Source) gplx.xowa.apps.servers.http.Http_server_wkr.Write_wiki(Unknown Source) gplx.xowa.apps.servers.http.Http_server_wkr.Process_get(Unknown Source) gplx.xowa.apps.servers.http.Http_server_wkr.Run(Unknown Source) gplx.xowa.apps.servers.http.Http_server_wkr.Invk(Unknown Source) gplx.Gfo_invk_.Invk_by_msg(Unknown Source) gplx.core.threads.Thread_adp.run(Unknown Source) java.lang.Thread.run(Thread.java:748)

Hmm, I see en.wikipedia.org-core.xowa is missing, so I'm assuming that April 2019 offline import I did is corrupt and needs to be redone. Is the script-based offline import process incremental and idempotent? With en.wikipedia.org-core.xowa missing, could I just run the offline import script again, or do I need to delete all the .xowa files in wiki/en.wikipedia.org before I run the script. (I'm restarting from scratch now, just curious for next time).

My goal is to have both en.wikipedia.org and en.wiktionary.org read-only sites with thumbnail images on a isolated network. Do I need to import the commons.wikimedia.org to achieve this?

Hey, thanks for the log posts. Replying to your questions.

Any hints on how to troubleshoot this?

Not really. As you say later, the core.xowa file is gone. I'm pretty sure XOWA would not have deleted it (XOWA does not delete database files, and it treats them mostly as read-only resources)

Do all wikis have to be imported in one operation? (Since it has been a couple of months since I did this, I'll probably redo the import procedure since my memory of the events is a bit fuzzy now).

Nope. Wikis can be imported individually (en.wikipedia.org first, then en.wiktionary.org a few days later, then fr.wikipedia.org a few hours later, etc)

Hmm, I see en.wikipedia.org-core.xowa is missing, so I'm assuming that April 2019 offline import I did is corrupt and needs to be redone.

That's strange. XOWA doesn't modify files after they're created (unless you edit the articles). However, if the core.xowa file is gone, then that would bork the entire en.wikipedia.org install.

Is the script-based offline import process incremental and idempotent? With en.wikipedia.org-core.xowa missing, could I just run the offline import script again, or do I need to delete all the .xowa files in wiki/en.wikipedia.org before I run the script. (I'm restarting from scratch now, just curious for next time).

You won't be able to re-run the offline import script and just regenerate that one core.xowa file. Among other things, the import process won't work if the directory is not empty.

In theory, the entire process is idempotent (same results generated if you run it again), and it is incremental (data is generated a piece at a time). However, for practical reasons, I don't support piece-wise regeneration of files. Also, the core.xowa file centralizes information from multiple processes, so the only way to "regen" the core.xowa file would be to re-run the entire process. At that point, you might as well scrap the existing files and start again.

My goal is to have both en.wikipedia.org and en.wiktionary.org read-only sites with thumbnail images on a isolated network. Do I need to import the commons.wikimedia.org to achieve this?

Unfortunately, XOWA just imports the wikitext for commons.wikimedia.org. It doesn't actually import all the files / images in commons.wikimedia.org. Keep in mind that there's roughly 22 TB of data, so it's not practical.

I'm uploading 2019-05 en.wikipedia.org now, so you can use that import for your enwiki thumbnails. There are older enwiktionary thumbnails at https://archive.org/download/Xowa_enwiki_latest (See the files titled Xowa_enwiktionary_2016-10_file). You can copy them over to your enwiktionary directory

Thanks for the replies; they all sound reasonable. I'm re-importing the 4-2019 dumps, for en.wiktionary, commons (92% done), en.wikipedia. Sounds like I should have skipped the commons import. I'll let them all finish just to confirm its working as expected and then I'll download and import 5-2019. Thanks so much for doing the 05-2019 thumbnail images!

Incidentally, I've been unable to get the offline import dialog to display the contents of the directory containing the articles.xml file. It just shows an empty file list pane. However, I can manually type in the path in the read from file field.

Thanks for the replies; they all sound reasonable. I'm re-importing the 4-2019 dumps, for en.wiktionary, commons (92% done), en.wikipedia. Sounds like I should have skipped the commons import. I'll let them all finish just to confirm its working as expected and then I'll download and import 5-2019. Thanks so much for doing the 05-2019 thumbnail images!

Cool. No problem!

Incidentally, I've been unable to get the offline import dialog to display the contents of the directory containing the articles.xml file. It just shows an empty file list pane. However, I can manually type in the path in the read from file field.

Hmmm... Maybe this is a 42.3 thing / your setup thing? I tried now with my openSUSE and it shows (though admittedly, the UI is pretty bad)

cat /etc/SuSE-release
openSUSE 42.2 (x86_64)
VERSION = 42.2                                                                                                                                                                                                                
CODENAME = Malachite

Screenshot_20190609_212917

Hmmm... Maybe this is a 42.3 thing / your setup thing? I tried now with my openSUSE and it shows (though admittedly, the UI is pretty bad)

I see exactly what you posted, but the righthand pane shows no files, i.e., no *pages-articles.xml.bz2

I've run into another issue: search doesn't work in http_server mode. Search works fine in gui mode. After entering the search term, I just get an empty Special:Search page. My first run had --http_server.special_pages_safelist XowaSearch but complained that search was missing. So then I passed it "XowaSearch|Search". That produced no error but an empty Special:Search page. Same empty page behavior if I run without the http_server.special_pages_safelist option. Below are the stdout and session log. No errors, they just stop short. My search term was "dog". I can then enter another search term and get the same result.

Interesting that this session.txt line doesn't show the "dog" search term: 20190612_210309.805 page.async: url=en.wikipedia.org/wiki/Special:Search?fulltext=y&search=

Search works great in http_server mode with v4.5.19.1801 and the 2017-12 data. Any ideas?

Will an offline import of the 2019-05 dump in v4.5.18.1801 along with the 2019-05 image function as a temporary work-around until this gets solved in 4.6.1.1903?

Thanks!

stdout

XOWA is licensed under the terms of the General Public License (GPL) Version 3, or alternatively under the terms of the Apache License Version 2.0.

You may use XOWA according to either of these licenses as is most appropriate for your project on a case-by-case basis.

The terms of each license can be found in the following files:

GPLv3 License: LICENSE-GPLv3.txt Apache License: LICENSE-APACHE2.txt

version: 4.6.1.1903; build date: 2019-03-27 04:39:22

arguments: [0] = '--wiki_dir' [1] = '/var/opt/xowa/wiki' [2] = '--app_mode' [3] = 'http_server' [4] = '--http_server_port' [5] = '80' [6] = '--http_server_home' [7] = 'en.wikipedia.org'

HTTP Server started: Navigate to http://localhost:80 wikipedia.local|GET|/ wikipedia.local|GET|/fsys/user/anonymous/wiki/en.wikipedia.org/html/xowa_common.css wikipedia.local|GET|/fsys/user/anonymous/wiki/en.wikipedia.org/html/xowa_wiki.css wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/popups/popups.css wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/core/core.css wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/core/core.js wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/search-suggest/search-suggest.css wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/core/exec.js wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/xoelem/xo.elem.js wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/core/DOMContentLoaded.js wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/search-suggest/search-suggest.js wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/find/xowa_find_html.js wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/find/xowa_find_html.css wikipedia.local|GET|/fsys/bin/any/xowa/html/res/lib/jquery/jquery-1.11.3.min.js wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/popups/xo.popup.js wikipedia.local|GET|/fsys/file/commons.wikimedia.org/thumb/d/d/9/2/Christa_Wolf%2C_Der_geteilte_Himmel_1963.jpg/120px.jpg wikipedia.local|GET|/fsys/file/commons.wikimedia.org/thumb/b/c/3/d/Z_domingo_savio.jpg/100px.jpg wikipedia.local|GET|/fsys/file/commons.wikimedia.org/thumb/4/a/6/9/Commons-logo.svg/31px.png wikipedia.local|GET|/fsys/file/commons.wikimedia.org/thumb/f/d/e/7/Pedro_S%C3%A1nchez2019%28portrait_crop%29.jpg/120px.jpg wikipedia.local|GET|/fsys/file/commons.wikimedia.org/thumb/3/d/0/c/Mediawiki-logo.png/35px.png wikipedia.local|GET|/fsys/file/commons.wikimedia.org/thumb/7/5/e/3/Wikimedia_Community_Logo.svg/35px.png wikipedia.local|GET|/fsys/file/commons.wikimedia.org/thumb/f/f/4/3/Wikidata-logo.svg/47px.png wikipedia.local|GET|/fsys/file/commons.wikimedia.org/thumb/f/a/2/6/Wikibooks-logo.svg/35px.png wikipedia.local|GET|/fsys/file/commons.wikimedia.org/thumb/f/a/6/c/Wikiquote-logo.svg/35px.png wikipedia.local|GET|/fsys/file/commons.wikimedia.org/thumb/2/4/8/c/Wikinews-logo.svg/51px.png wikipedia.local|GET|/fsys/file/commons.wikimedia.org/thumb/4/c/7/1/Wikisource-logo.svg/35px.png wikipedia.local|GET|/fsys/file/commons.wikimedia.org/thumb/d/f/a/e/Wikispecies-logo.svg/35px.png wikipedia.local|GET|/fsys/file/commons.wikimedia.org/thumb/0/b/4/1/Wikiversity_logo_2017.svg/41px.png wikipedia.local|GET|/fsys/file/commons.wikimedia.org/thumb/e/c/5/3/Wiktionary-logo.svg/35px.png wikipedia.local|GET|/fsys/file/commons.wikimedia.org/thumb/d/d/0/0/Wikivoyage-Logo-v3-icon.svg/35px.png wikipedia.local|GET|/fsys//bin/any/xowa/html/css/footer/wikimedia-button.png wikipedia.local|GET|/fsys//bin/any/xowa/html/css/footer/poweredby_mediawiki_88x31.png wikipedia.local|GET|/fsys/bin/any/xowa/file/app.general/twisty_down.png wikipedia.local|GET|/fsys/user/anonymous/wiki/en.wikipedia.org/html/en.wikipedia.org/static/images/mobile/copyright/wikipedia-wordmark-en.svg wikipedia.local|GET|/fsys/user/anonymous/wiki/en.wikipedia.org/html/logo.png wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/gadgets/navframe/mediawiki.gadget.navframe.js wikipedia.local|GET|/en.wikipedia.org/wiki/Special:XowaSearch?search=dog&fulltext=y wikipedia.local|GET|/fsys/user/anonymous/wiki/en.wikipedia.org/html/xowa_common.css wikipedia.local|GET|/fsys/user/anonymous/wiki/en.wikipedia.org/html/xowa_wiki.css wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/core/core.css wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/popups/popups.css wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/core/core.js wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/core/exec.js wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/search-suggest/search-suggest.css wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/core/DOMContentLoaded.js wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/find/xowa_find_html.css wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/xoelem/xo.elem.js wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/search-suggest/search-suggest.js wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/find/xowa_find_html.js wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/xowa/popups/xo.popup.js wikipedia.local|GET|/fsys/bin/any/xowa/html/res/lib/jquery/jquery-1.11.3.min.js wikipedia.local|GET|/fsys//bin/any/xowa/html/css/footer/wikimedia-button.png wikipedia.local|GET|/fsys//bin/any/xowa/html/css/footer/poweredby_mediawiki_88x31.png wikipedia.local|GET|/fsys/bin/any/xowa/file/app.general/twisty_down.png wikipedia.local|GET|/fsys/user/anonymous/wiki/en.wikipedia.org/html/logo.png wikipedia.local|GET|/fsys/user/anonymous/wiki/en.wikipedia.org/html/en.wikipedia.org/static/images/mobile/copyright/wikipedia-wordmark-en.svg wikipedia.local|GET|/fsys/bin/any/xowa/html/res/src/gadgets/navframe/mediawiki.gadget.navframe.js

session.txt

20190612_210156.439 env.init: version=4.6.1.1903 20190612_210156.483 env.init: jar_url=xowa_linux_64.jar; build_date=2019-03-27 04:39:22 20190612_210156.484 env.init: op_sys=linux64 20190612_210156.489 arguments: [0] = '--wiki_dir' [1] = '/var/opt/xowa/wiki' [2] = '--app_mode' [3] = 'http_server' [4] = '--http_server_port' [5] = '80' [6] = '--http_server_home' [7] = 'en.wikipedia.org'

20190612_210156.606 app.ctor.bgn 20190612_210156.794 app.ctor.end 20190612_210156.888 app.boot:servers 20190612_210156.889 app.boot:app.init 20190612_210157.092 db.schema.load.bgn: conn=data source=/var/opt/xowa/user/xowa.user.anonymous.sqlite3;version=3; 20190612_210157.282 db.schema.load.bgn: conn=data source=/var/opt/xowa/bin/any/xowa/addon/app/cfg/xo.cfg.sqlite3;version=3; 20190612_210157.369 app.boot:gfs.run 20190612_210157.389 gfs.done: /var/opt/xowa/user/anonymous/app/cfg/os.linux_64.gfs 20190612_210157.426 gfs.done: /var/opt/xowa/bin/any/xowa/cfg/app/xowa.gfs 20190612_210157.427 app.boot:app.launch 20190612_210157.432 wiki.db_core.v2.standard: url=/var/opt/xowa/bin/any/xowa/wiki/home/home.xowa 20190612_210157.444 db.schema.load.bgn: conn=data source=/var/opt/xowa/bin/any/xowa/wiki/home/home.xowa;version=3; 20190612_210157.456 fsdb.db_core.v2: type=all url=/var/opt/xowa/bin/any/xowa/wiki/home/home.xowa 20190612_210157.431 wiki.init.bgn: home 20190612_210157.616 wiki.init.end 20190612_210157.625 site_cfg loaded; wiki=home itm=extensiontags tid=itm 20190612_210157.630 site_cfg loaded; wiki=home itm=interwikimap tid=itm 20190612_210157.654 HTTP Server started: Navigate to http://localhost:80 20190612_210233.801 wiki.db_core.v2.standard: url=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-core.xowa 20190612_210233.803 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-core.xowa;version=3; 20190612_210233.806 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000.xowa;version=3; 20190612_210233.808 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.004.xowa;version=3; 20190612_210233.809 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.100.xowa;version=3; 20190612_210233.811 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.012.xowa;version=3; 20190612_210233.812 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.118.xowa;version=3; 20190612_210233.814 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.006.xowa;version=3; 20190612_210233.815 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.010.xowa;version=3; 20190612_210233.817 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.008.xowa;version=3; 20190612_210233.818 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.002.xowa;version=3; 20190612_210233.820 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.014.xowa;version=3; 20190612_210233.821 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.003.xowa;version=3; 20190612_210233.827 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.004.xowa;version=3; 20190612_210233.828 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.828.xowa;version=3; 20190612_210233.829 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.005.xowa;version=3; 20190612_210233.831 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.006.xowa;version=3; 20190612_210233.833 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.108.xowa;version=3; 20190612_210233.834 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.007.xowa;version=3; 20190612_210233.835 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.004-db.002.xowa;version=3; 20190612_210233.836 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.008.xowa;version=3; 20190612_210233.838 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.710.xowa;version=3; 20190612_210233.839 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.009.xowa;version=3; 20190612_210233.840 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.2300.xowa;version=3; 20190612_210233.841 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.010.xowa;version=3; 20190612_210233.843 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.004-db.003.xowa;version=3; 20190612_210233.844 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.011.xowa;version=3; 20190612_210233.846 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-xtn.search.core.xowa;version=3; 20190612_210233.847 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-xtn.search.link-title-ns.000-db.001.xowa;version=3; 20190612_210233.849 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-xtn.search.link-title-ns.999-db.001.xowa;version=3; 20190612_210233.850 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-xtn.category.core.xowa;version=3; 20190612_210233.852 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-xtn.category.link-db.001.xowa;version=3; 20190612_210233.853 fsdb.db_core.v2: type=lot url=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-file-core.xowa 20190612_210233.779 wiki.init.bgn: en.wikipedia.org 20190612_210233.887 wiki.init.end 20190612_210233.888 site_cfg loaded; wiki=en.wikipedia.org itm=extensiontags tid=cfg 20190612_210233.889 site_cfg loaded; wiki=en.wikipedia.org itm=interwikimap tid=cfg 20190612_210233.911 page.load: url=en.wikipedia.org/wiki/Main Page 20190612_210234.899 page_load: loaded wikitext; page=Main_Page wikitext_len=6712 20190612_210234.997 jtidy.init; elapsed=36 20190612_210235.198 wiki.init.bgn: www.wikidata.org 20190612_210235.207 wiki.init.end 20190612_210235.211 page.async: url=en.wikipedia.org/wiki/Main Page 20190612_210235.213 redlink.redlink_bgn: page=Main Page total_links=98 20190612_210235.223 page.load: url=en.wikipedia.org/wiki/Main Page 20190612_210235.245 redlink.redlink_end: redlinks_run=3 20190612_210235.722 page_load: loaded wikitext; page=Main_Page wikitext_len=6712 20190612_210309.783 page.load: url=en.wikipedia.org/wiki/Special:XowaSearch 20190612_210309.788 page_load: loaded wikitext; page=Special:XowaSearch wikitext_len=0 20190612_210309.805 page.async: url=en.wikipedia.org/wiki/Special:Search?fulltext=y&search= 20190612_210309.807 redlink.redlink_bgn: page=Special:XowaSearch total_links=0 20190612_210309.807 redlink.redlink_end: redlinks_run=0 20190612_210309.807 page.load: url=en.wikipedia.org/wiki/Special:XowaSearch 20190612_210309.809 page_load: loaded wikitext; page=Special:XowaSearch wikitext_len=0

I did an offline import of the 2019-05 enwiki and enwiktionary dumps. Then I copied the en.wikipedia.org-file-ns.X-db.Y.xowa files to the site directory. This is on OpenSuSE 42.3, java-1_8_0-openjdk-headless-1.8.0.181-27.1.x86_64

Search for "dog" now works under 4.5.19.1801, but when I click on the resulting top link, "Dog", xowa errors out and never returns a response. Seems like the structure of the Wikipedia dumps changed between 2017-12 and 2019-05. Did it?

However, if I run in gui mode on an internet connected machine, it seems to work fine. I did a search for dog and cat and was able to view the results both times. (Does xowa use its local data in gui mode? It seems to fetch a lot from the internet.) Then I switched to http_server mode with no internet connection. A search for rat returns results, but when I click on the top hit, Rat, xowa errors out. After that, xowa is no longer responsive to the browser: Firefox 60.2.2esr.

Xowa 4.5.19.1801 works much better with the 2017-12 data on OpenSuSE 42.2 with java-1_8_0-openjdk-headless-1.8.0.151-10.18.2.x86_64. However its interesting that the search for Dog with the 2019-05 data returns the page Dog as the top hit but with the 2017-12 data, the top hit is "Dog Eat Dog" and "Dog" doesn't even appear in the results.

How well is http_server mode working for you without an internet connection?

Here are the logs for 4.5.18.1801, http_server mode with 2019-05 offline imported data.

err.txt:

20190613_113721.125 cfg:itm not found; key=xowa.gui.toolbar.show_search 20190613_113721.127 cfg:itm not found; key=xowa.gui.toolbar.show_allpages 20190613_113841.159 invoke failed: Dog {{#invoke:Citation/CS1|citation |CitationClass=journal }} [err 0] =Module:Citation/CS1/Identifiers:44 attempt to index ? (a nil value) stack traceback: Module:Citation/CS1/Identifiers:23: in function <Module:Citation/CS1/Identifiers:22> Module:Citation/CS1/Identifiers:575: in function <Module:Citation/CS1/Identifiers:574> Module:Citation/CS1/Identifiers:1003: in function <Module:Citation/CS1/Identifiers:1002>

internet.txt: 20190613_114102.445 download fail: src='https://commons.wikimedia.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:Full-protection-shackle.svg' trg='mem/download.tmp' error='Connection timed out (Connection timed out)' 20190613_114309.676 download fail: src='https://en.wikipedia.org/w/api.php?action=query&format=xml&prop=imageinfo&iiprop=size|url&redirects&titles=File:Full-protection-shackle.svg' trg='mem/download.tmp' error='Connection timed out (Connection timed out)'

session.txt: 20190613_113633.505 env.init: version=4.5.19.1801 20190613_113633.556 env.init: jar_url=xowa_linux_64.jar; build_date=2018-01-06 06:39:00 20190613_113633.556 env.init: op_sys=linux64 20190613_113633.561 arguments: [0] = '--wiki_dir' [1] = '/var/opt/xowa/wiki' [2] = '--app_mode' [3] = 'http_server' [4] = '--http_server_port' [5] = '80' [6] = '--http_server.special_pages_safelist' [7] = 'XowaSearch|Search' [8] = '--http_server_home' [9] = 'en.wikipedia.org'

20190613_113633.679 app.ctor.bgn 20190613_113633.863 app.ctor.end 20190613_113634.053 app.boot:servers 20190613_113634.053 app.boot:app.init 20190613_113635.246 db.schema.load.bgn: conn=data source=/var/opt/xowa/user/xowa.user.anonymous.sqlite3;version=3; 20190613_113635.763 db.schema.load.bgn: conn=data source=/var/opt/xowa/bin/any/xowa/addon/app/cfg/xo.cfg.sqlite3;version=3; 20190613_113635.910 app.boot:gfs.run 20190613_113636.029 gfs.done: /var/opt/xowa/user/anonymous/app/cfg/os.linux_64.gfs 20190613_113636.170 gfs.done: /var/opt/xowa/bin/any/xowa/cfg/app/xowa.gfs 20190613_113636.170 app.boot:app.launch 20190613_113636.192 wiki.db_core.v2.standard: url=/var/opt/xowa/bin/any/xowa/wiki/home/home.xowa 20190613_113636.234 db.schema.load.bgn: conn=data source=/var/opt/xowa/bin/any/xowa/wiki/home/home.xowa;version=3; 20190613_113636.273 fsdb.db_core.v2: type=all url=/var/opt/xowa/bin/any/xowa/wiki/home/home.xowa 20190613_113636.175 wiki.init.bgn: home 20190613_113636.494 wiki.init.end 20190613_113636.540 site_cfg loaded; wiki=home itm=extensiontags tid=itm 20190613_113636.545 site_cfg loaded; wiki=home itm=interwikimap tid=itm 20190613_113636.569 HTTP Server started: Navigate to http://localhost:80 20190613_113721.125 cfg:itm not found; key=xowa.gui.toolbar.show_search 20190613_113721.127 cfg:itm not found; key=xowa.gui.toolbar.show_allpages 20190613_113721.447 wiki.db_core.v2.standard: url=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-core.xowa 20190613_113721.521 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-core.xowa;version=3; 20190613_113721.541 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000.xowa;version=3; 20190613_113721.546 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.004.xowa;version=3; 20190613_113721.557 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.100.xowa;version=3; 20190613_113721.561 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.012.xowa;version=3; 20190613_113721.585 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.118.xowa;version=3; 20190613_113721.598 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.006.xowa;version=3; 20190613_113721.602 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.010.xowa;version=3; 20190613_113721.607 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.008.xowa;version=3; 20190613_113721.620 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.002.xowa;version=3; 20190613_113721.633 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.014.xowa;version=3; 20190613_113721.645 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.003.xowa;version=3; 20190613_113721.667 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.004.xowa;version=3; 20190613_113721.672 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.828.xowa;version=3; 20190613_113721.675 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.005.xowa;version=3; 20190613_113721.688 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.006.xowa;version=3; 20190613_113721.691 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.108.xowa;version=3; 20190613_113721.712 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.007.xowa;version=3; 20190613_113721.728 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.004-db.002.xowa;version=3; 20190613_113721.748 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.008.xowa;version=3; 20190613_113721.765 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.710.xowa;version=3; 20190613_113721.781 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.009.xowa;version=3; 20190613_113721.784 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.2300.xowa;version=3; 20190613_113721.788 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.010.xowa;version=3; 20190613_113721.803 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.004-db.003.xowa;version=3; 20190613_113721.807 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-text-ns.000-db.011.xowa;version=3; 20190613_113721.814 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-xtn.search.core.xowa;version=3; 20190613_113721.824 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-xtn.search.link-title-ns.000-db.001.xowa;version=3; 20190613_113721.826 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-xtn.search.link-title-ns.999-db.001.xowa;version=3; 20190613_113721.828 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-xtn.category.core.xowa;version=3; 20190613_113721.844 db.schema.load.bgn: conn=data source=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-xtn.category.link-db.001.xowa;version=3; 20190613_113721.846 fsdb.db_core.v2: type=lot url=/var/opt/xowa/wiki/en.wikipedia.org/en.wikipedia.org-file-core.xowa 20190613_113721.366 wiki.init.bgn: en.wikipedia.org 20190613_113722.212 wiki.init.end 20190613_113722.213 site_cfg loaded; wiki=en.wikipedia.org itm=extensiontags tid=cfg 20190613_113722.214 site_cfg loaded; wiki=en.wikipedia.org itm=interwikimap tid=cfg 20190613_113722.309 page.load: url=en.wikipedia.org/wiki/Main Page 20190613_113726.101 page_load: loaded wikitext; page=Main_Page wikitext_len=6712 20190613_113726.544 jtidy.init; elapsed=80 20190613_113728.152 wiki.init.bgn: www.wikidata.org 20190613_113728.189 wiki.init.end 20190613_113728.193 page.async: url=en.wikipedia.org/wiki/Main Page 20190613_113728.196 redlink.redlink_bgn: page=Main Page total_links=96 20190613_113728.242 page.load: url=en.wikipedia.org/wiki/Main Page 20190613_113730.091 page_load: loaded wikitext; page=Main_Page wikitext_len=6712 20190613_113731.616 redlink.redlink_end: redlinks_run=3 20190613_113801.365 page.load: url=en.wikipedia.org/wiki/Special:XowaSearch?search=dog&fulltext=y 20190613_113801.406 search.search by page_tbl; search=dog 20190613_113801.452 search.search by link_tbl; search=dog 20190613_113801.455 search.resolving; phrase=dog score_bgn=999815 score_end=1000001 20190613_113807.426 search.detaching; phrase=dog score_bgn=951455 score_end=975263 20190613_113807.461 page_load: loaded wikitext; page=Special:XowaSearch wikitext_len=34831 20190613_113807.511 page.async: url=en.wikipedia.org/wiki/Special:Search?fulltext=y&search=dog 20190613_113807.512 redlink.redlink_bgn: page=Special:XowaSearch total_links=0 20190613_113807.512 redlink.redlink_end: redlinks_run=0 20190613_113807.514 page.load: url=en.wikipedia.org/wiki/Special:XowaSearch?search=dog&fulltext=y 20190613_113807.515 search.search by page_tbl; search=dog 20190613_113807.516 search.search by link_tbl; search=dog 20190613_113807.516 search.detaching; phrase=dog score_bgn=951455 score_end=975263 20190613_113807.532 page_load: loaded wikitext; page=Special:XowaSearch wikitext_len=34831 20190613_113837.333 page.load: url=en.wikipedia.org/wiki/Dog 20190613_113841.153 qid not found in wikidata; qid=Q131276 page=Dog 20190613_113841.159 invoke failed: Dog {{#invoke:Citation/CS1|citation |CitationClass=journal }} [err 0] =Module:Citation/CS1/Identifiers:44 attempt to index ? (a nil value) stack traceback: Module:Citation/CS1/Identifiers:23: in function <Module:Citation/CS1/Identifiers:22> Module:Citation/CS1/Identifiers:575: in function <Module:Citation/CS1/Identifiers:574> Module:Citation/CS1/Identifiers:1003: in function <Module:Citation/CS1/Identifiers:1002> Module:Citation/CS1:2148: in function <Module:Citation/CS1:2143> Module:Citation/CS1:3637: in function <Module:Citation/CS1:3636> mw.lua:531: in function

Sorry. As I said elsewhere, my days have been long this week.

Responding piecemeal:

I see exactly what you posted, but the righthand pane shows no files, i.e., no *pages-articles.xml.bz2

Oops. I see what you mean. This must be an SWT / KDE thing. It works fine on Windows. Let me look into it later.

I've run into another issue: search doesn't work in http_server mode. Search works fine in gui mode. After entering the search term, I just get an empty Special:Search page. My first run had --http_server.special_pages_safelist XowaSearch but complained that search was missing. So then I passed it "XowaSearch|Search". That produced no error but an empty Special:Search page. Same empty page behavior if I run without the http_server.special_pages_safelist option. Below are the stdout and session log. No errors, they just stop short. My search term was "dog". I can then enter another search term and get the same result.

Yeah, this is a known issue. desb42@ had a fix for this, but I haven't incorporated it yet. See #264

Will an offline import of the 2019-05 dump in v4.5.18.1801 along with the 2019-05 image function as a temporary work-around until this gets solved in 4.6.1.1903?

I'll try to fix HTTP search in the near future. I'm still working through the backlog of tickets. :(

I did an offline import of the 2019-05 enwiki and enwiktionary dumps. Then I copied the en.wikipedia.org-file-ns.X-db.Y.xowa files to the site directory. This is on OpenSuSE 42.3, java-1_8_0-openjdk-headless-1.8.0.181-27.1.x86_64

Search for "dog" now works under 4.5.19.1801, but when I click on the resulting top link, "Dog", xowa errors out and never returns a response. Seems like the structure of the Wikipedia dumps changed between 2017-12 and 2019-05. Did it?

4.5.19.1801 probably can't handle 2019-05 English Wikipedia. You should use 4.6.1.1903: https://github.com/gnosygnu/xowa/releases/tag/v4.6.1.1903 . Also, I'll be release a new version within a week or so.

However, if I run in gui mode on an internet connected machine, it seems to work fine. I did a search for dog and cat and was able to view the results both times.

That's odd. Both GUI and HTTP Server should work the same for article retrieval. For http_server, does it work if you just go directly to the page? For example, http://localhost:8080/en.wikipedia.org/wiki/Dog

(Does xowa use its local data in gui mode? It seems to fetch a lot from the internet.)

By default, it tries to connect to the internet for images. You can disable it by unchecking "Web access enabled" here: home/wiki/Special:XowaCfg?grp=xowa.app.security

Then I switched to http_server mode with no internet connection. A search for rat returns results, but when I click on the top hit, Rat, xowa errors out. After that, xowa is no longer responsive to the browser: Firefox 60.2.2esr.

Yeah, this sounds like an http_server or bug #264 above. Can you try with v4.6.1.1903?

Xowa 4.5.19.1801 works much better with the 2017-12 data on OpenSuSE 42.2 with java-1_8_0-openjdk-headless-1.8.0.151-10.18.2.x86_64. However its interesting that the search for Dog with the 2019-05 data returns the page Dog as the top hit but with the 2017-12 data, the top hit is "Dog Eat Dog" and "Dog" doesn't even appear in the results.

That's odd, but it may have to do with how you're mixing the databases. Try again with 2017-12 data, but also go to home/wiki/Special:XowaDiag?type=fs.check&wiki=en.wikipedia.org . Paste the results here (I just want to see which databases)

How well is http_server mode working for you without an internet connection?

Honestly, I use XOWA gui when I'm on the desktop and XOWA Android for my phone.

Here are the logs for 4.5.18.1801, http_server mode with 2019-05 offline imported data.

Yeah, that looks fine at first glance. Try again with v4.6.1.1903 and paste logs here if it still fails?

Thanks.

Just to clarify, I first experienced the search failure with v4.6.1.1903 and the 2019-05 data. That is what the first set of logs represent. I tried falling back to v4.5.18.1801 with the 2019-05 data in the hopes that the search bug was simply a regression between v4.5.18.1801 and v4.6.1.1903. I didn't realize that the structure of the Wikipedia data/dumps is still evolving. So v4.5.18.1801 in http_server mode with 2017-12 data will remain very useful until search works with v4.6.x with more recent data. Are you asking me to test v4.6.1 http_server mode search with 2017-12 data?

Are you asking me to test v4.6.1 http_server mode search with 2017-12 data?

Oops, sorry. Didn't realize. No need to test v4.6.1 http_server mode search with 2017-12 data.

I tried now, and there are definitely issues with the search data for 2019-05. Let me investigate more and see if I can up load a fix. Thanks.

Although the v4.6.2 release notes didn't mention the http_server mode search issue, I tried it anyway with the 2019-05 en.wikipedia data and the behavior is the same as v4.6.1. Thanks for the other fixes though!

Yeah, the problem is that the 2019-05 search indices were corrupted. I need to build a non-corrupted version, but this is blocked by #483

With reference to #483, I did an offline import of 2019-05 on 6/10/2019 and my notes say it took just under 3 hours (from unpacked dumps - takes 52 minutes to unpack). I did this on an OpenSuSE 42.3 VM with 16G RAM, 8 cores, java-1_8_0-openjdk-headless-1.8.0.181-27.1.x86_64.

// import wiki from dump file app.setup.cmds.cmd_add("wiki.dump_file", "/var/opt/wikisrc/enwiktionary-20190501-pages-articles.xml", "en.wiktionary.org", "");

// import wiki from dump file app.setup.cmds.cmd_add("wiki.dump_file", "/var/opt/wikisrc/enwiki-20190501-pages-articles.xml", "en.wikipedia.org", "");

I'll try it again today with 4.6.2. Also, the http_server mode search behaves the same with en.wiktionary.org.

The 4.6.2 offline import of both en.wiktionary and en.wikipedia completed in 2 hours 48 minutes. Search works fine on both in GUI mode, but neither in http_server mode. The search in wiktionary produced the following trace:

failed to process request; request=type: GET url: /fsys/user/anonymous/wiki/en.wiktionary.org/html/xowa_common.css protocol: HTTP/1.1 host: localhost user_agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Firefox/60.0 accept: text/css,/;q=0.1 accept_encoding: gzip, deflate dnt: false x_requested_with: <> cookie: <> referer: http://localhost/en.wiktionary.org/wiki/Special:XowaSearch?search=dog&fulltext=y content_length: 0 content_type: <> content_type_boundary: <> connection: keep-alive pragma: <> cache_control: <> errmsg=[err 0] null [err 1] invalid UTF-8 sequence: s=<> [trace]: gplx.Bry.new_u8(Unknown Source) gplx.core.consoles.Console_adpsys.Clear_tmp(Unknown Source) gplx.core.consoles.Console_adpsys.Write_str_w_nl(Unknown Source) gplx.core.net.Http_server_wtr__console.Write_str_w_nl(Unknown Source) gplx.xowa.apps.servers.http.Http_server_wkr.Process_get(Unknown Source) gplx.xowa.apps.servers.http.Http_server_wkr.Run(Unknown Source) gplx.xowa.apps.servers.http.Http_server_wkr.Invk(Unknown Source) gplx.Gfoinvk.Invk_by_msg(Unknown Source) gplx.core.threads.Thread_adp.run(Unknown Source) java.lang.Thread.run(Thread.java:748)

With reference to #483, I did an offline import of 2019-05 on 6/10/2019 and my notes say it took just under 3 hours (from unpacked dumps - takes 52 minutes to unpack).

Yeah, sorry about not explaining this.

There are 2 types of builds:

Wikitext build: This takes the Wikimedia dump and generates the .text.xowa databases. It also does some simple indexing. This is the build you did and will generally complete in about 3 hours
HTML build: This takes the .text.xowa databases and generates .html.xowa databases. It also builds more sophisticated indexing (Page Rank) as well as the full-text index. This build takes somewhere around 10 days now (see #483).

When I said this:

Yeah, the problem is that the 2019-05 search indices were corrupted.

I meant that #483 corrupted the 2019-05 search indices and whatever I uploaded to archive.org wasn't working.

In addition, I've been negligent about #264, so the HTTP_Server search doesn't work.

That said, I'm not getting the errors you are, so maybe something else is wrong. Can you try http://localhost/en.wiktionary.org/wiki/Special:XowaSearch?search=dog&fulltext=y with the latest version of XOWA (https://github.com/gnosygnu/xowa/releases/tag/v4.6.2.1907)? If you still see the same error in the logs, we'll have to do more investigation from there.

Thanks.

That URL returns no result, just an empty page with the heading Special:XowaSearch

XOWA is licensed under the terms of the General Public License (GPL) Version 3, or alternatively under the terms of the Apache License Version 2.0.

You may use XOWA according to either of these licenses as is most appropriate for your project on a case-by-case basis.

The terms of each license can be found in the following files:

GPLv3 License: LICENSE-GPLv3.txt Apache License: LICENSE-APACHE2.txt

version: 4.6.2.1907; build date: 2019-07-25 04:59:58

arguments: [0] = '--wiki_dir' [1] = '/var/opt/xowa/wiki' [2] = '--app_mode' [3] = 'http_server' [4] = '--http_server_port' [5] = '80' [6] = '--http_server.special_pages_safelist' [7] = 'XowaSearch|Search' [8] = '--http_server_home' [9] = 'en.wikipedia.org'

HTTP Server started: Navigate to http://localhost:80 localhost|GET|/en.wiktionary.org/wiki/Special:XowaSearch?search=dog&fulltext=y cfg:itm not found; key=xowa.gui.toolbar.show_search cfg:itm not found; key=xowa.gui.toolbar.show_allpages localhost|GET|/fsys/user/anonymous/wiki/en.wiktionary.org/html/xowa_wiki.css localhost|GET|/fsys/user/anonymous/wiki/en.wiktionary.org/html/xowa_common.css localhost|GET|/fsys/bin/any/xowa/html/res/src/xowa/core/core.css localhost|GET|/fsys/bin/any/xowa/html/res/src/xowa/search-suggest/search-suggest.css localhost|GET|/fsys/bin/any/xowa/html/res/src/xowa/popups/popups.css localhost|GET|/fsys/bin/any/xowa/html/res/src/xowa/core/exec.js localhost|GET|/fsys/bin/any/xowa/html/res/src/xowa/core/core.js localhost|GET|/fsys/bin/any/xowa/html/res/src/xowa/core/DOMContentLoaded.js localhost|GET|/fsys/bin/any/xowa/html/res/src/xowa/xoelem/xo.elem.js localhost|GET|/fsys/bin/any/xowa/html/res/src/xowa/search-suggest/search-suggest.js localhost|GET|/fsys/bin/any/xowa/html/res/src/xowa/find/xowa_find_html.css localhost|GET|/fsys/bin/any/xowa/html/res/src/xowa/find/xowa_find_html.js localhost|GET|/fsys/bin/any/xowa/html/res/lib/jquery/jquery-1.11.3.min.js localhost|GET|/fsys/bin/any/xowa/html/res/src/xowa/popups/xo.popup.js localhost|GET|/fsys//bin/any/xowa/html/css/footer/wikimedia-button.png localhost|GET|/fsys/bin/any/xowa/html/res/src/gadgets/navframe/mediawiki.gadget.navframe.js localhost|GET|/fsys//bin/any/xowa/html/css/footer/poweredby_mediawiki_88x31.png localhost|GET|/fsys/bin/any/xowa/file/app.general/twisty_down.png localhost|GET|/fsys/user/anonymous/wiki/en.wiktionary.org/html/logo.png localhost|GET|/fsys/bin/any/xowa/file/app.window/app_icon.png

Thanks for the logs

I'm working on a fix for this weekend such that urls like /en.wiktionary.org/wiki/Special:XowaSearch?search=dog&fulltext=y will return results (and won't be a blank page)
I'm also going to try getting the search suggestions working (desb42 had gotten this working on his side, so this should be possible)
I won't be able to handle the full-text search. This needs a full build. I'll try to build one in the next few weeks

I'll post again here when ready. Thanks again.

Hi!

I fixed this in tonight's release. See https://github.com/gnosygnu/xowa/releases/tag/v4.6.3.1908

Let me know if you run into any issues. Otherwise, I'll close this out in the next few days.

Thanks!

This seems to solve search for older e.wikipedia.org versions as well. Thank you!

Wow, this is awesome!!!! With the initial testing that I've done so far, it works great! Thank you so much!!! I am thoroughly delighted!!!

@Muurtegel / @ktry : Cool. Good to hear. Thanks!

gnosygnu / xowa

Search: Search not working for 2019-05 English Wikipedia? [Multiple Wikis] #489

stdout

session.txt