search

gleanerio / scheduler

Scheduling approaches related to gleaner tooling
Apache License 2.0
1 stars 4 forks source link

SHACL - with Dask run #37

Open valentinedwv opened 1 year ago

valentinedwv commented 1 year ago

Blog on how to run DASK task, Since doug used Dask to run SHACL, posting:

https://dagster.io/blog/dagster-dask

fils commented 1 year ago

Nice.. will definitely read.

Since I tend to use Dask to speed up work across cores on my local machine, vs the way cooler multi-machine manner in which Dask really shines, I have thought about trying out: https://github.com/sybrenjansen/mpire

I should try and compare them for SHACL or other tests.

Lately I've been playing with pulling data to feed into LLMs, which might be a fun plae to compare them too.