search

haskell / haskell-wiki-configuration

Issue tracking for Haskell Wiki
https://wiki.haskell.org/
4 stars 4 forks source link

Request: start HaskellWiki automation #23

Open Anton-Latukha opened 5 years ago

Anton-Latukha commented 5 years ago

wiki-scripts for wiki platform maintanance and server-side linting:

  1. Code
  2. Docs
  3. Additional info how it operates on the platform

Scribunto with Lua for scripting inside MediaWiki:

  1. Extension:Scribunto
  2. Lua scripting introduction
  3. Lua scripting tutorial
  4. Scribunto/Lua reference manual

wiki-monkey for client-side linting/editing help (useful for users and maintainers):

  1. Repository and Wiki and for users the Standalone version installation,
  2. (optional) Server-side database and Client-server version

ArchWiki:

  1. Code

Preface

Automation is a great way to save a lot of effort and time. Especially in the maintenance of the Wiki. Stopping and doing automation is way better than trying to achieve it, while also keeping doing manual day-to-day wiki tasks.

Maybe automation would also allow to solve and open HaskellWiki registration.

Here is the story about that, with useful links, the most useful references gathered above.

Story of quality

ArchWiki is one of the most active and polished technical wikis in the IT world. Indeed, in the Linux segment it is currently the biggest in active (contributing) users (source: WikiApiary), with honorable mentions of Gentoo (with ... ooh, shiny new look; with way more in pages) and Fedora Project.

Story of what quality costed

Back in the days, I perceived the shear volume of work the core team of maintainers (6) was keeping up with to hold wiki concise and polished. Every day they reviewed and made a huge volume of edits, discussions, maintaining the Wiki.

I back then also talked with them sharing my ideas on style/documentation policies, how to simplify things for them, and users, and what can be automated in linting wiki syntax, scripting "magic words".

Back then kynikos already implemented wiki-monkey. Which I already been using (if only every wiki docs contributor used it). They ran this client-side bot by loading/editing in the browser (I remember it was described as being done almost manually) for every page. At the time they said they only distantly heard about server-side scripting inside Wikipedia. And said that lahwaacz started trying to make some brushstrokes of maintenance scripts himself (wiki-scripts). They were so busy, that could not do anything else, and also to help me to implement server-side scripting (various linting tasks) (I was requesting give some needed IT support to make it), and there was only sole server instance setup - so they said that I need to pursue aim myself. I asked for a configuration code, and was not provided (install was said only partly documented and to be of no use for me, and no access to the sole and main server or getting a testing sample from it). Information was scarce and terse, and I even wrote to and found one of the developers of that scripting and mailed with him. I tried, but of course, from outside, could not closely replicate, get and migrate a huge chunk of wikitext to agree with my part-replica to massive test for their particular wiki instance, for my results to be of importance to them. It was impossible task without their help.

Here now I also found archwiki code.

Story of success

Wiki syntax has special cases and some clunkiness due to it being the first of a kind, but it is also really future rich and flexible. Automation on server&user-side makes much easier to maintain the quality of the Wiki.

From that time of a story, both wiki-monkey and wiki-scripts progressed far. Their solutions can be seen as a success in the maintenance of a Wiki. Moreover then ArchWiki is a very technical place.

And also now I see that there is much more and of better quality documentation about Lua scripting is available now on MediaWiki.

Anton-Latukha commented 5 years ago

Just tested wiki-monkey currently works great on ArchWiki. On Wikipedia it does not show up (it must show-up on edit pages).

Anton-Latukha commented 5 years ago

Now it even does what we discussed previously. It finds relevant external wiki links and converts them to interwiki links:

[https://en.wikipedia.org/wiki/Line_discipline] -> [[wikipedia:Line_discipline]]
Anton-Latukha commented 5 years ago

Troubleshooted what is wrong with Wikipedia. Console log:

Content Security Policy: The page’s settings observed the loading of a resource at https://cdn.rawgit.com/kynikos/wiki-monkey/v5.0.1/dist/WikiMonkey-ArchWiki.min.js (“script-src”). A CSP report is being sent.
...
Unhandled promise rejection Error: "Unknown dependency: mediawiki.api.edit"

Sadly wiki-monkey seems so unknown and unused by people that Wikipedia doesn't have it whitelisted in security filters.

Anton-Latukha commented 5 years ago

When I tried Wikipedia profile of script on HaskellWiki: Console log:

Use of Mutation Events is deprecated. Use MutationObserver instead. content.js:30:251
hgolden commented 5 years ago

Anton, I think all of these ideas should be implemented.

First, though, we should upgrade the wiki to the latest MediaWiki LTS (#20). I would appreciate your help in doing this. Please let us know if you are interested in working on this.

Anton-Latukha commented 5 years ago

Seems like my lazy computation was not pure enough, and returned effect =)

Intriguing proposition. I had a good thought about this.

I can do something really close to described below closer to mid-summer, end of summer, early autumn.

My current plan is: I still have to catch NixOS guys after release & work with them to merge work that was ready & tested for 1.5 years https://github.com/NixOS/nix/pull/1565. I still have to finish https://github.com/syl20bnr/spacemacs/pull/10977 after these couple of month, and it may involve some Elisp and Clojure.

My movement aligned to finish basic training in Haskell:

And then finish basic training by contributing to https://github.com/haskell-nix/hnix.

Then I would go to work in some paid business. Maybe I would need somehow to migrate to the Netherlands to save personal life.

Regarding Wiki work

I have some DevOps skills.

And after mid-summer, the things would be clearer.

I would be interested first to:

  1. Backups & Replicas.
  2. lift configuration into containerized paradigm with Docker images. And hear me out.
  3. Then even on one server even with some Compose they can be scaled/have some switching/rollback capability.
  4. Then use Dockerfiles and images to lift them into small Kubernetes cluster (even if 3 VMs inside one instance) for even more flexibility and the possibility of reliable redundancy and vertical scalability. Then even from one host - Kubernetes cluster can grow to any number of provided nodes. Have some blue&green deployments.
  5. Then I'd go to the guys that deployed NixOS Wiki and ask them around about how they deployed it, their deployment code on NixOS&Nix, what issues they are having.
  6. Then think about does current MediaWiki in NixOS can be vertically scalable.
  7. And then migrate Dockerfiles & Kubernetes setup (which would be basic anyway) into Nix codebase and NixOS containers.

Then - the trick is done, HaskellWiki is bootstrapped with Haskell infrastructure. Maintaining HaskellWiki with hnix can be achieved, in the future of Haskell&Nix. Because of the surrounding community, this architecture would be for a long time to come, there would be people pleased and interested in it, people would themselves submit suggestions&changes, and become contributors and maintainers of external Wiki infrastructure, so that is assured. In that Wiki software would be kept up-to-date. So contributors to infrastructure and internal Wiki contributors would become interested more in internal affairs. With public & documented repositories of internal automation scripts - more activity can be drawn to lint&maintain internals and less maintenance and more automation happen.

If something, I can hook-up in any of the stages, if the process would not wait for me.

And on internal automation:

  1. In the early stages of migration - during work fix wiki-monkey and deploy it.
  2. Then - during further work - understand wiki-scripts maintenance scripts. They are important ones to maintain internal infrastructure.
  3. Ask questions to ArchWiki team.
  4. After all that - automation knowledge can be shared to NixOS Wiki and others.
  5. Automation scripts ideas&code shared between highly-technical Wikis.