search

sfu-natlang / lensingwikipedia

Lensing Wikipedia is an interface to visually browse through human history as represented in Wikipedia. This the source code that runs the website:
http://lensingwikipedia.cs.sfu.ca
Other
11 stars 4 forks source link

Crawler crashed #184

Open msiahbani opened 9 years ago

msiahbani commented 9 years ago

Due to some bugs in python scrapy, the data-preparation does not work any more. I'll try to fix it.

anoopsarkar commented 9 years ago

We should really move to wikipedia dump and only process delta changes from the previous day.

anoopsarkar commented 9 years ago

Which is the last full crawl that has a complete annotation? I tried the latest one with a fullData.json file but it had no Person or Location entities.

msiahbani commented 9 years ago

the last full crawl with complete annotation : /cs/natlang-projects/users/maryam/wikiCrawler/Crawl_20150202/fullData.json

I am working on the issue today. There is an issue with python libraries.

On Fri, Jul 24, 2015 at 4:44 PM, Anoop Sarkar notifications@github.com wrote:

Which is the last full crawl that has a complete annotation? I tried the latest one with a fullData.json file but it had no Person or Location entities.

— Reply to this email directly or view it on GitHub https://github.com/sfu-natlang/lensingwikipedia/issues/184#issuecomment-124716653 .

msiahbani commented 9 years ago

The issues with python libraries for scrapy are fixed now.

But there is an issue with SRL. I had to restart the server, as it did not accept any client. Now It just finish the first sentence of the given file. I am still working on it.

anoopsarkar commented 9 years ago

ok. thanks. maybe Ramtin can help with updating the modules if that is the problem.

anoopsarkar commented 8 years ago

is the Perl issue fixed? Either @ramtinms or I can help fix the system issues.

msiahbani commented 8 years ago

I am not sure what the problem is. I can run UIUC SRL server but cannot query it.

On Tue, Aug 25, 2015 at 9:20 AM, Anoop Sarkar notifications@github.com wrote:

is the Perl issue fixed? Either @ramtinms https://github.com/ramtinms or I can help fix the system issues.

— Reply to this email directly or view it on GitHub https://github.com/sfu-natlang/lensingwikipedia/issues/184#issuecomment-134657618 .

anoopsarkar commented 8 years ago

let's try to fix this before mid next week if at all possible.