search

visualfabriq / bquery

A query and aggregation framework for Bcolz (W2013-01)
https://www.visualfabriq.com
BSD 3-Clause "New" or "Revised" License
56 stars 11 forks source link

Add benchmark results to README? #7

Closed shoyer closed 9 years ago

shoyer commented 9 years ago

This would be nice to see just to get a sense of what bquery can do without installing it first. Thanks! This looks like a potentially very cool project.

FrancescElies commented 9 years ago

Hi, thanks for your suggestion, we'll write them there. In the future it would be nice to have an ipython notebook showing the results, something similar to what bcolz has (http://nbviewer.ipython.org/github/Blosc/movielens-bench/blob/master/querying-ep14.ipynb)

FrancescElies commented 9 years ago

Hi Stephan, now you should be able to see results in the readme file, btw there was a mistake in the previous benchmark which it made the result from pandas go slower than it really was (my fault)

CarstVaartjes commented 9 years ago

I've also added some more info around the project, but in general on-disk bcolz aggregations are 1.5-3.0x slower than in-memory Pandas at the moment. For us it's a huge benefit as the difference in speed is not that bad (and will only get better) while solving a huge amount of issues around in-memory caching, hot sets, etc.

shoyer commented 9 years ago

Thanks! I agree, in-memory pandas is not really the competition here but it's nice to see that you're not too far off.