search

visualfabriq / bquery

A query and aggregation framework for Bcolz (W2013-01)
https://www.visualfabriq.com
BSD 3-Clause "New" or "Revised" License
56 stars 11 forks source link

Multithreaded v2 #38

Closed FrancescElies closed 7 years ago

FrancescElies commented 9 years ago

This is another tryout, based on master.

So far no performance improvements, I guess something like https://github.com/Blosc/bcolz/pull/160 is needed to have bigger sections wit gil released

FrancescElies commented 7 years ago

closing

CarstVaartjes commented 7 years ago

Hi Francesc!

How are you? In the end we made this (also open source): https://github.com/visualfabriq/bqueryd It's a zeromq based, distributed network over which you can spread "bcolz shards" and which can massively parallellize bcolz queries. Final code should be released next week I guess!

FrancescElies commented 7 years ago

Hey Carst, I'm good, and you? zeromq and bcolz looks like a marriage made in heaven to me. I'll have a look at bqueryd as soon as I get some free time, I'll try to keep up to date. Good luck with the new release next week!