can't access to Chinese wiki pages

densitydesign / strumentalia-seealsology

see also section scraping on custom levels of depth

Other

83 stars 24 forks source link

can't access to Chinese wiki pages #30

Closed ericYH01 closed 2 years ago

ericYH01 commented 2 years ago

I have tried to crawl the Chinese wiki, but it returns an error msg as follows, "Warning: these are not valid wikipedia pages: https://zh.wikipedia.org/zh-tw/%E7%9F%A5%E8%AF%86"

I also tried another format, and the same error occurred, "https://zh.wikipedia.org/zh-tw/知識" : "Warning: these are not valid wikipedia pages: https://zh.wikipedia.org/zh-tw/知識"

Would it be possible to support such web pages? Thanks for advice.

boogheta commented 2 years ago

Hello, there was indeed a problem with the language specific pages such as /zh-tw/ This is now fixed, although chinese pages are not handled yet and should be added as explained in the yellow infobox, please do so if you can! thanks.