brucemcpherson / desktopliberation

hosting for desktop liberation google plus community
30 stars 1 forks source link

Blistering fast file streaming between Drive and Cloud Storage using Cloud Run #95

Open brucemcpherson opened 3 years ago

brucemcpherson commented 3 years ago

A friend of mine hit an Apps Script problem the other day when transferring large amounts of data between Drive and Cloud Storage. We’ve all had the mysterious ‘unexpected Javacript runtime error’ from the IDE that comes and goes, and it’s down to a number of things, one of which appears to be UrlFetch silently timing out.

I wondered if there might be a better way to solve this problem using Cloud Run to be directed to make cloud transfers. Of course this means that Apps Script isn’t really involved at all, except to make the initial cloud run request, and if Apps Script isn’t involved it means we can also do streaming transfers rather than heave around large POST bodies.

The solution I came up with works on node, from the cli or from any app that can make a post really. https://ramblings.mcpher.com/gcp/file-drive-and-cloud-storage/