The wikicrawler does not include the encoding into the html document. Therefore, Internet Explorer and Firefox do not display special characters correctly. It works fine with chrome.
adding the following tag to the head solves the issue:
<metacharset="utf-8">
The wikicrawler does not include the encoding into the html document. Therefore, Internet Explorer and Firefox do not display special characters correctly. It works fine with chrome.
adding the following tag to the head solves the issue:
<meta
charset="utf-8">
We need to include this into the python script.