2020 DE-EN Wikipedia(?) + How's it going?

[Ope30]

What's up! How's life? Is everything okay? I know it's a little late, but all the best for the new year 2020!

It's been a long time since I last created a issue. Nevertheless I always thought about XOWA and often asked myself how the development of XOWA looks like now. So I was thinking, how about I say hello! :)

Like I said, I've always thought about XOWA, the thing is I've been very busy with myself, but now I have more time. Another good thing to mention is that I now have much more storage space. I don't even know what to do with all that storage space honestly, but then comes XOWA, because it's the only thing I like to sacrifice my storage space for!

I must have missed a lot. There's probably an easier way to download Wikipedia yourself now than before, or is it still the same? I tried to download the Wikipedias myself (that was a long time ago) so as not to bother you, but I must honestly confess that the process was quite complicated. But that may have changed now. I think with the current knowledge I might be able to overcome it myself.

Either way, I'd love to make it happen this month. I plan to distance myself from the Internet for a certain period of time and, thanks to XOWA, carry a lot of knowledge with me!

[gnosygnu]

Hey, happy new year! Sorry for the lack of responses, but I haven't had much time recently.

With regards to new wikis, I hit a snag a few months ago due to my build machine going down. I finally manged to get some time to rebuild it today, so will be working on building 2020-01 English / German wikis. desb42 has been amazing with finding bugs, so hopefully these builds will be much better than last year's.

Will be working on this over the next few weeks, and update as I make progress. Thanks!

[Ope30]

Sorry for the lack of responses, but I haven't had much time recently.

No worries! Pretty understandable.

With regards to new wikis, I hit a snag a few months ago due to my build machine going down. I finally manged to get some time to rebuild it today, so will be working on building 2020-01 English / German wikis. desb42 has been amazing with finding bugs, so hopefully these builds will be much better than last year's.

Will be working on this over the next few weeks, and update as I make progress. Thanks!

I'm glad to hear that! Thank you for taking the time for this. I appreciate it. Yeah, thanks to desb42 an incredible number of bugs have been fixed. You two together have done a professional job. 💪

[gnosygnu]

Just a quick comment for others

I believe I have bad memory sticks on my Linux box. I've ordered new ones, so this may take a few weeks. Please bear with the technical difficulties.

Sorry.

[Ope30]

I believe I have bad memory sticks on my Linux box. I've ordered new ones, so this may take a few weeks. Please bear with the technical difficulties.

Okay.

FYI: Lately, I've really been spending a lot of time with XOWA, and I've stumbled upon some, I hate to say it, bugs, that I thought it would be beneficial for you to know about. I have been using German 2019-03 and English 2019-05. I don't know if it is worthwhile to fix all the bugs because they are present in older builds (2019-03, 2019-05)? If you can and want to, I can also open a new issue when/if the wikis 2020-01 are accessible and explicitly report everything I can find to improve the usability.

Either way, if you also need someone to test stuff like new XOWA builds, testing articles, testing new functions, new wiki builds, HTTP server, etc, I would be glad to participate! As far as programming is concerned, I can unfortunately not make myself useful. But I know how to get on the right track and make myself useful through the right use of XOWA. :)

I adapt to you. Just bring it to my attention if you need anything. I would try to make myself useful. XOWA is a great project. It is the best program of its kind, in my solemn opinion even to this day!

[moralesalberto]

Hello, hope everything is going well. Let me know if there is anything I can help with this Jan 2020 export.

[gnosygnu]

Thanks for the bump. My Linux box has had issues, and I may have to replace some hardware. Also, I'm getting a bunch of missing images.

I'm kicking off a 2020-02 German Wiki build and see how that goes

[gnosygnu]

2020-02 German Wiki has been uploaded. See https://archive.org/download/Xowa_dewiki_latest

I'm working on 2020-02 English Wiki, but that will take a week at least

[Ope30]

2020-02 German Wiki has been uploaded. See https://archive.org/download/Xowa_dewiki_latest

I'm working on 2020-02 English Wiki, but that will take a week at least

Thank you so much for taking the time to build and upload German Wikipedia! Downloading right now.

And by the way, I just stopped the import for English Wikipedia. I feel safer if you do that with all your experience, if you know what I mean. Just to be on the safe side. Thank you for taking the time for this as well!

[gnosygnu]

I built 2020-02 English Wikipedia. It has a few minor issues (broken refs on London), but I'm going to start uploading sometime Monday. Will post again when done.

[Ope30]

I built 2020-02 English Wikipedia. It has a few minor issues (broken refs on London), but I'm going to start uploading sometime Monday. Will post again when done.

As always, I can't thank you enough for taking the time to create and upload the wikis!

I think I know what you mean by broken refs. I discovered them in the German Wikipedia 2020-02. I also discovered a few other things. All this can be (so to speak) fixed by playing around with Scribunto, e.g. by changing luaj to lua. Or by changing the tidy engine from jtidy to tidy. But this will only fix number 1 and break number 2. Personally, I don't mind that. I'm not at my PC at the moment (I type this with my tablet because my PC has no internet), so I can't give more details. As soon as 2020-02 EN is uploaded, I will create a new issue and show you exactly what I mean.

By the way, I imported Wikidata 2020-02 with the offline dumps a few days ago. It took me about 2 days. Everything is working. 👍🏼

[gnosygnu]

archive.org keeps failing for me after 57 GB.

I started a new upload under https://archive.org/details/Xowa_enwiki_2020-02 (link is currently not working). It's at 25 GB of 175GB and probably won't be done for at least another week

[Ope30]

archive.org keeps failing for me after 57 GB.

I started a new upload under https://archive.org/details/Xowa_enwiki_2020-02 (link is currently not working). It's at 25 GB of 175GB and probably won't be done for at least another week

Damnit! That sucks. I think archive.org is the best site to upload stuff. Hopefully everything is okay with archive.org. Maybe they added some new rules regarding upload policies? No idea...

Thanks for creating a new link!

[gnosygnu]

Just an update. At 94 GB of 175GB. Still failing regularly, but past 57 GB at least

[Ope30]

Just an update. At 94 GB of 175GB. Still failing regularly, but past 57 GB at least

Okay. No problem. This shouldn’t take long. Thanks for the info.

[gnosygnu]

Finally finished uploading: https://archive.org/details/Xowa_enwiki_2020-02

[Ope30]

Finally finished uploading: https://archive.org/details/Xowa_enwiki_2020-02

That's incredible. Thanks again for taking the time to upload English Wikipedia! To be honest, I'm glad that's over now. Now you can concentrate on other, more important things. I've already tested the wiki, and everything is fine except for a few script errors there and a few other little things there. No big change to the game. I will show you what I meant by broken refs, I will create a new issue and show you what I mean. 👍🏼

Thanks!

[gnosygnu]

Cool. Thanks for the quick download!

Will reply to your other issue but moving this to done for now. Thanks!