search

dnmilne / wikipediaminer

An open source toolkit for mining Wikipedia
130 stars 62 forks source link

Wikpedia Miner as web service slowness #31

Open baderex opened 8 years ago

baderex commented 8 years ago

Hello,

I'm using Wikpedia Miner as a web service with Tomcat. The problem is that it takes too much time to process http request. On average, 3 requests/minute and I have about 75K requests which will take over 2 weeks to finish with this performance. What can be done to speed things up?

My machine is an i7 with 4GB ram.

Thanks

Neuw84 commented 8 years ago

With 4GB impossible, if you put more memory you can cache more databases in order to improve the speed.

El 12 nov. 2016 9:31 PM, "baderex" notifications@github.com escribió:

Hello,

I'm using Wikpedia Miner as a web service with Tomcat. The problem is that it takes too much time to process http request. On average, 3 requests/minute and I have about 75K requests which will take over 2 weeks to finish with this performance. What can be done to speed things up?

My machine is an i7 with 4GB ram.

Thanks

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/dnmilne/wikipediaminer/issues/31, or mute the thread https://github.com/notifications/unsubscribe-auth/AFpYAiKs_BvP_S7uR4H6bOfS84DdCDk7ks5q9iIsgaJpZM4KwhOt .