search

tenex / opensourcecontributors

Find all contributions for a user through the GitHub Archive
91 stars 11 forks source link

Cache results #16

Closed fatso83 closed 9 years ago

fatso83 commented 9 years ago

Compared to the enormous amount of data you are working with I would assume that the (presumably) little traffic that's hitting you would be manageable to cache for each user (to some extent).

For interested developers that would poll your site regularly, this should lower the load quite a bit.

hut8 commented 9 years ago

Mongo seems to cache it for me.

If you request this: http://githubcontributions.io/user/fatso83

or: http://githubcontributions.io/user/fatso83/events

Then request it again, isn't it instant the second time?

hut8 commented 9 years ago

Also if you have an idea of a more intelligent way to do this, I'm all ears. I was actually thinking of making a .json.bz2 for each user and doing away with any sort of real database. Even though this seems insane, some early results were promising, except that compiling those files would take months initially (benchmarked with a magnetic-drive VPS and ext4), and appending to each would require unzipping each so that it would have a chance to re-compress it efficiently.

fatso83 commented 9 years ago

I have no idea why I did not respond to this, but yeah, it was instant on refresh.