search

datapartnership / datapartnership.github.io

:pencil: Projects & News
https://datapartnership.org
Other
6 stars 0 forks source link

Dask vs Spark for Large Geospatial Processing #31

Closed MrMaksimize closed 2 years ago

mireille-raad commented 4 years ago

should i be commenting here? I can't resist saying that I have thoughts on this!! I did this once for dask vs spark for large processing but minus the geo-spatial bit.

I hope the post is public, because I want to read it :)

g4brielvs commented 4 years ago

@mireille-raad Absolutely! You are more than welcome 😄

I had some experiences with both Spark and Dask. You definitely can help us. What are your thoughts on that?

mireille-raad commented 4 years ago

i can help review the post if you want, especially if it is public, because that counts as public knowledge. For more time consuming things, I am sure I can't really "work" on this :)

Holly-Transport commented 4 years ago

👍

On Mon, Jun 15, 2020 at 11:17 AM Mireille Raad notifications@github.com wrote:

i can help review the post if you want, especially if it is public, because that counts as public knowledge. For more time consuming things, I am sure I can't really "work" on this :)

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/datapartnership/datapartnership.org/issues/31#issuecomment-644198506, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABCI5PXGF47BYKIHRJ4M2ZLRWY3SFANCNFSM4N6H4CAA .

brunosan commented 4 years ago

A very recent discovery I'm very impressed with is the pangeo+dask ecosystem (+binder for 1-click deployments)

Basically pangeo+dask is explained in 2min here https://www.youtube.com/watch?v=rSOJKbfNBNk

MrMaksimize commented 4 years ago

should i be commenting here? I can't resist saying that I have thoughts on this!! I did this once for dask vs spark for large processing but minus the geo-spatial bit.

I hope the post is public, because I want to read it :)

@mireille-raad - I want to hear your thoughts!! You should definitely be commenting here :)