search

WikiTeam / wikiteam

Tools for downloading and preserving wikis. We archive wikis, from Wikipedia to tiniest wikis. As of 2023, WikiTeam has preserved more than 350,000 wikis.
https://github.com/WikiTeam
GNU General Public License v3.0
705 stars 147 forks source link

"Probably a loop, finishing" in wiki-commons.genealogy.net #126

Open nemobis opened 10 years ago

nemobis commented 10 years ago

Got that line while getting titles from wiki-commons.genealogy.net, is all right?

emijrp commented 10 years ago

This is the fix #100

Remove these lines and you probably will see an infinite loop retrieving titles:

        if len(titles) != len(set(titles)):
            #probably we are in a loop, server returning dupe titles, stop it
            print 'Probably a loop, finishing'
            titles = list(set(titles))
            apfrom = ''
nemobis commented 10 years ago

What should I do in such cases? May I be missing some titles?

PiRSquared17 commented 9 years ago

It's possible, but it would be more likely that it is in a loop.