search

BrightstarDB / BrightstarDB

This is the core development repository for BrightstarDB.
http://brightstardb.com/
MIT License
458 stars 81 forks source link

Streaming Queries #112

Open kal opened 10 years ago

kal commented 10 years ago

Investigate the possibility of supporting a subset of SPARQL for streaming query processing.

rvesse commented 10 years ago

I've started work in the dotNetRDF 1.9 branch on a streaming query engine. Things are at a fairly early stage and the API is changing significantly to allow this but it is looking promising and I've run some basic queries end to end.

Obviously not everything will stream because some operators require keeping large amounts of data in-memory e.g. DISTINCT and ORDER BY but for the most part things will be streaming. Plus the new API should make it much easier to extend/swap out specific parts of the implementation e.g. I already have pluggable join strategies working.

It is nowhere near ready for production usage yet and there are lots of parts of SPARQL not yet implemented but something to be aware of. As for ETA probably towards the end of the year if I am being realistic (if not longer) but there will likely be early access Alphas available in the nearer term for experimentation purposes

kal commented 10 years ago

Thanks for the info Rob. I'll probably look at doing this as part of B* 2.0 development then.

rvesse commented 10 years ago

No problem, shall we plan on getting together at some point in the Autumn to discuss this in person?

kal commented 10 years ago

Yeah, that would be great - maybe Oct/Nov timeframe? Lets get in touch a bit closer to the time.