search

dnmilne / wikipediaminer

An open source toolkit for mining Wikipedia
130 stars 62 forks source link

Investigate more sophisticated caches, like EHCache or Guava Cache. #10

Open dnmilne opened 10 years ago

dnmilne commented 10 years ago

Angel, can you please describe what we would get by using EHCache? My take is that we don't need many features from the cache, except that it needs to be fast, small, and simple (just key lookup, no traversing, etc)?

Neuw84 commented 10 years ago

Hi,

We could tune the performance very easily (EHcache has a .xml file for configuration), add distributed caching, caching to memory, to hard disk,configure persistent caching, (not loading the cache every time you launch the application...)... and more things.

Your approach is perfect for the task as no more is needed. But maybe it's worth trying to integrate some of this caches for scaling reasons and see the performance penalty (should't be too much).

An option could be added for letting the user this decision, use your approach for performance and simplicity or use other cache mechanism for tuning how, how much and where the things are cached.

What do you think?