openaustralia / yinyo

A wonderfully simple API driven service to reliably execute many long running scrapers in a super scaleable way
https://yinyo.io
Apache License 2.0
6 stars 1 forks source link

Allow builds to use more memory than runs #166

Open mlandauer opened 4 years ago

mlandauer commented 4 years ago

In many cases, like ruby scrapers, the build phase uses a lot of memory while the runs don’t use much at all. Also the build is usually much quicker than the run. So this is all quite inefficient