search

dask / dask-ec2

Start a cluster in EC2 for dask.distributed
106 stars 37 forks source link

Change default volume size to 8GB #65

Open stumitchell opened 7 years ago

stumitchell commented 7 years ago

The default 500GB is far to large in my opinion and leads to large EBS charges.

The new 8GB default should be enough for most workloads an matches what is the default on EC2

mrocklin commented 7 years ago

Dask uses disk to spill excess data. I would expect it to be at least as large as RAM, if not larger. I don't know what level is correct, but 8GB seems small to me.

quasiben commented 7 years ago

500 is perhaps too large a default setting but I agree, 8GBs is a bit too small. What do you think about 100GB ?

stumitchell commented 7 years ago

yeah I'm not sure what your use cases are but a default setting of 100GB would have definitely made my EBS bill alot smaller :-)