gene1wood commented 2 years ago

URL to the Wiki page

Many URLs

What's the issue?

It looks like there was a bug in GitHub's wiki system a while back that caused wiki page deletions to not remove the page from the search index. As a result, when you search the gluetun wiki, there are many pages that are returned which don't exist.

You can see many of them using this search

https://github.com/qdm12/gluetun/search?o=asc&q=the&s=updated&type=wikis

This post says that the fix for the bug for these old deleted wiki pages is to recreate the page and then delete it again.

https://github.com/orgs/community/discussions/22085#discussioncomment-3235485

Below is the list of wiki pages that I found in search but which don't exist.

Likely the easiest way to do this is not through the web UI but directly with the GitHub git repo that backs the wiki.

The steps would be something like this

git clone git@github.com:qdm12/gluetun.wiki.git
cd gluetun.wiki
touch "32-bit-setup.md"
touch "Advanced.md"
touch "Commands.md"
touch "Connect-to-gluetun.md"
touch "Cyberghost-Servers.md"
touch "DNS-over-TLS.md"
touch "Docker-image.md"
touch "External-firewall-requirements.md"
touch "Fastestvpn-Servers.md"
touch "Guidelines.md"
touch "HideMyAss-Servers.md"
touch "Mullvad-IPv6.md"
touch "Niche-features.md"
touch "Openvpn.md"
touch "Openvpn-file.md"
touch "Ping-timeout.md"
touch "Portainer-iptables-error.md"
touch "Privado-Servers.md"
touch "Private-Internet-Access-Servers.md"
touch "Privatevpn-Servers.md"
touch "Purevpn-Servers.md"
touch "Servers-Readme.md"
touch "Setup.md"
touch "Surfshark-Servers.md"
touch "Synology.md"
touch "Synology-setup.md"
touch "Testing.md"
touch "Testing-the-setup.md"
touch "Torguard-Servers.md"
touch "Uncommon-issues.md"
touch "Update-servers-information.md"
touch "Updating.md"
touch "Vyprvpn-Servers.md"
git commit -m "Adding temp wiki pages to work around GitHub bug https://github.com/orgs/community/discussions/22085#discussioncomment-3235485"
rm -f "32-bit-setup.md"
rm -f "Advanced.md"
rm -f "Commands.md"
rm -f "Connect-to-gluetun.md"
rm -f "Cyberghost-Servers.md"
rm -f "DNS-over-TLS.md"
rm -f "Docker-image.md"
rm -f "External-firewall-requirements.md"
rm -f "Fastestvpn-Servers.md"
rm -f "Guidelines.md"
rm -f "HideMyAss-Servers.md"
rm -f "Mullvad-IPv6.md"
rm -f "Niche-features.md"
rm -f "Openvpn.md"
rm -f "Openvpn-file.md"
rm -f "Ping-timeout.md"
rm -f "Portainer-iptables-error.md"
rm -f "Privado-Servers.md"
rm -f "Private-Internet-Access-Servers.md"
rm -f "Privatevpn-Servers.md"
rm -f "Purevpn-Servers.md"
rm -f "Servers-Readme.md"
rm -f "Setup.md"
rm -f "Surfshark-Servers.md"
rm -f "Synology.md"
rm -f "Synology-setup.md"
rm -f "Testing.md"
rm -f "Testing-the-setup.md"
rm -f "Torguard-Servers.md"
rm -f "Uncommon-issues.md"
rm -f "Update-servers-information.md"
rm -f "Updating.md"
rm -f "Vyprvpn-Servers.md"
git commit -m "Removing temp wiki pages to work around GitHub bug https://github.com/orgs/community/discussions/22085#discussioncomment-3235485"
git push

Here's the list of pages that show up in search but which don't exist

qdm12 commented 2 years ago

Awesome, thanks for the copy pasta commands! 👍

Search looks like it's giving less results now. Let me know if you think it's fixed now? 😉

qdm12 commented 2 years ago

I think it's fixed now. Anyway thanks again for the help and taking the time to create the detailed issue 💯

gene1wood commented 2 years ago

@qdm12 No problem, thank you for making this great docker container. Least I can do.

gene1wood commented 2 years ago

@qdm12 Hmm it looks like it didn't work.

Doing a search for all wiki pages returns 110 pages but there are only 65 actual pages in the wiki.

Also, that search still shows ones like Commands.

Could you try creating empty pages for all of those ghost wiki pages, pushing it up to the wiki, letting it sit for a day for GitHub to notice and index the new empty pages, then as a second step, commit a deletion?

I did a complete listing of every wiki page in the search cache and did a set comparison with the contents of the wiki. There are 33 pages (total) present in the cache that are missing from the wiki.

Would you do these two steps on two different days from a fresh clone of the wiki?

Day 1

git clone git@github.com:qdm12/gluetun.wiki.git
cd gluetun.wiki
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "32 bit setup.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Advanced.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Commands.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Connect to gluetun.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Cyberghost Servers.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "DNS over TLS.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Docker image.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Fastestvpn Servers.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Guidelines.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "HideMyAss Servers.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Mullvad IPv6.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Mullvad Servers.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Openvpn file.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Openvpn.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Ping timeout.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Portainer iptables error.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Privado Servers.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Private Internet Access Servers.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Privatevpn Servers.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Protonvpn Servers.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Purevpn Servers.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Servers Readme.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Setup.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Surfshark Servers.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Synology setup.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Synology.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Testing the setup.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Testing.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Torguard Servers.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Updating Servers.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Updating.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Vyprvpn Servers.md"
echo "Disregard this wiki page. It will only be here temporarily while we clear out garbage from GitHub's search cache" > "Windscribe Servers.md"
git commit -m "Adding temp wiki pages to work around GitHub bug https://github.com/orgs/community/discussions/22085#discussioncomment-3235485"
git push

Day 2

cd gluetun.wiki
git pull
rm -f "32 bit setup.md"
rm -f "Advanced.md"
rm -f "Commands.md"
rm -f "Connect to gluetun.md"
rm -f "Cyberghost Servers.md"
rm -f "DNS over TLS.md"
rm -f "Docker image.md"
rm -f "Fastestvpn Servers.md"
rm -f "Guidelines.md"
rm -f "HideMyAss Servers.md"
rm -f "Mullvad IPv6.md"
rm -f "Mullvad Servers.md"
rm -f "Openvpn file.md"
rm -f "Openvpn.md"
rm -f "Ping timeout.md"
rm -f "Portainer iptables error.md"
rm -f "Privado Servers.md"
rm -f "Private Internet Access Servers.md"
rm -f "Privatevpn Servers.md"
rm -f "Protonvpn Servers.md"
rm -f "Purevpn Servers.md"
rm -f "Servers Readme.md"
rm -f "Setup.md"
rm -f "Surfshark Servers.md"
rm -f "Synology setup.md"
rm -f "Synology.md"
rm -f "Testing the setup.md"
rm -f "Testing.md"
rm -f "Torguard Servers.md"
rm -f "Updating Servers.md"
rm -f "Updating.md"
rm -f "Vyprvpn Servers.md"
rm -f "Windscribe Servers.md"
git commit -m "Removing temp wiki pages to work around GitHub bug https://github.com/orgs/community/discussions/22085#discussioncomment-3235485"
git push

qdm12 commented 2 years ago

I pushed the 'day 1' files yesterday, do you see any improvement? I got 144 results 🤔

gene1wood commented 2 years ago

@qdm12 Yes, I also see 144. But importantly, I see the new updated pages in the index. Would you go ahead and execute the Day 2 steps to see if that clears all those pages from the index? (what a pain)

qdm12 commented 2 years ago

Done, let's see what happens like tomorrow. Really disappointing Github 😢 I guess I should work on #579 sooner rather than later.

gene1wood commented 2 years ago

Maddening. Looks like the search index has reverted to the contents from before the temp files (I can't even imagine how this is possible).

Ok, I guess we just give up?

Search index still showing original file contents

qdm12 commented 2 years ago

😆 Yeah... Well let me try to nuke it and restore it see if this helps.

qdm12 commented 2 years ago

Ok so I tried force pushing all the commits in a single squashed commit. Let's wait tomorrow, if it doesn't work, yeah give up and move away from this (sadly) terrible wiki system.

qdm12 commented 2 years ago

Yeah it doesn't work still, I have restored the git history and dropped the commits trying to fix it 😢 I'll try to move to another wiki system soon. Let's close this for now.

qdm12 / gluetun

Wiki issue: Corrupted GitHub search index of wiki pages #1102

URL to the Wiki page

What's the issue?

Day 1

Day 2