search

ipeirotis / WikiSynonyms

Extracts synonyms for various terms, exploiting the redirects between terms in Wikipedia
http://wikisynonyms.ipeirotis.com/
12 stars 3 forks source link

Refresh periodically Wikipedia entries #17

Open ipeirotis opened 11 years ago

ipeirotis commented 11 years ago

Fetch periodically the Wikipedia dumps and refresh the database (every week? every month?)

Examine whether it is feasible to launch on the spot an EC2 instance that will handle the task of updating the database, and then we can mark the service as down for maintenance while the update is running

georgegg commented 11 years ago

Need to discuss what approach we'll follow. As you know this process consumes a lot of resources (CPU, and Disk space). The deploy server you suggest on issue #16 has very little free space. See readme on Build section. We need at least 10GB of free space for the cron job.

I need to know of alternatives depending on how fast we need this concerning our timetable and cost for Amazon EC2 and RDS.